kaleko / courseraml Goto Github PK

Hello
In Ex 3, Neural networks, in the part with multiclass classification I think there is a problem with regularization in Logistic regression problem. I am talking particularly about part : 1.3 Vectorizing Logistic Regression.

right_hand = mytheta.T.dot( mytheta ) * mylambda / (2*m)
Here the first element of theta (theta_0 if you'd like) is also included in regularization. It should not be included.

There seems to be a subtle difference between your normalization function and the original Matlab version. As a result, the unrated zeros in myY become negative when Ymean is subtracted from them.

def normalizeRatings(myY, myR):
    ...
    Ymean = np.sum(myY,axis=1)/np.sum(myR,axis=1)  # Unrated elements are ignored here...
    Ymean = Ymean.reshape((Ymean.shape[0],1))

    # ...but not here. The zero elements in myY become negative after subtracting Ymean
    return myY-Ymean, Ymean

I think this will yield odd results and possibly prevent convergence. One way to address this would be to also zero out the unrated elements after the subtraction:

    return np.multiply(myY-Ymean, myR), Ymean

Hello,

I would like to know the license of the code in the repository.

Thanks,
Nicolás

Thx for your sharing

When I review ext1, I have a question about below code

        tmptheta[j] = theta[j] - (alpha/m)*np.sum((h(initial_theta,X) - y)*np.array(X[:,j]).reshape(m,1))

here why always use initial_theta instead of newest "theta"

I was able to reproduce the graph of the two principal components, but they were not orthogonal. Prinicpal components should be orthogonal (pc1.dot(pc2) == 0) no?

Edit: I think it doesn't look perpendicular/orthogonal due to rounding.

in the last of the file,
sorted_indices = np.argsort( linear_svm.coef_, axis=None )[::-1]
should be
sorted_indices = np.argsort( linear_svm.coef_, axis=None )[::-1] + 1
beause argsort values start at 0

Hi,

Great work !,
Im gonna do something on similar lines but less pythonic ,
the only thing i wanna use are your plot modules (To plot the data as im a noob in matplot lib),
i will give credits but still do i have your permission to include the portions of your code?

It would save me a lot of time by doing so . :)

the line
tmptheta = theta
should be updated to
tmptheta = np.copy(theta) // create a deep copy of theta instead of a new reference!

Otherwise the below statement will update theta also while the tmptheta is being updated (as both are references to same object!). theta should NOT change during an iteration of gradient descent!
tmptheta[j] = theta[j] - (alpha/m)*np.sum((h(theta,X) - y)*np.array(X[:,j]).reshape(m,1))

When running the "def plotBoundary(mytheta, myX, myy, mylambda=0.):"

I got the following error: "contour.py:1173: UserWarning: No contour levels were found within the data range."

Is there a bug in the code or is it a mistake on my side?

Thanks!

There is list that name is 'thetahistory' in function 'descendGradient'.

if you print it, you can find out all of element in thetahistory are same.
so graph is like

after change code like

thetahistory.append(list(theta[:,0]))

therefore, final graph is below

ex1 folder, ex1.ipynb file, the method descentGradient,when compute the tmptheta[j], it should be tmptheta[j] = theta[j] - (alpha/m)*np.sum((h(theta,X)-y)*np.array(X[:,j]).reshape(m,1)), instead of current code tmptheta[j] = theta[j] - (alpha/m)*np.sum((h(**initial_**theta,X)-y)*np.array(X[:,j]).reshape(m,1))

Hello, Thanks for your helps! It's awesome guide to me and really helpful.
If I would like to make suggest or find error, Is it possible to commit?
If it is possible, How can I commit? just clone & make new branch & commit?

Thanks