tirthajyoti / machine-learning-with-python Goto Github PK

I see you are writing Python code for optimization on GitHub. A general problem for gradient descent and other non-linear algorithms - particularly for more complex equations - is the choice of initial parameters to start the "descent" in error space. Without good starting parameters, the algorithm will stop in a local error minimum. For this reason the authors of scipy have added a genetic algorithm for initial parameter estimation for use in gradient descent. The module is named scipy.optimize.differential_evolution.

I have used scipy's Differential Evolution genetic algorithm to determine initial parameters for fitting a double Lorentzian peak equation to Raman spectroscopy of carbon nanotubes and found that the results were excellent. The GitHub project, with a test spectroscopy data file, is:

https://github.com/zunzun/RamanSpectroscopyFit

If you have any questions, please let me know. My background is in nuclear engineering and industrial radiation physics, and I love Python, so I will be glad to help.

Thought it may be helpful/convenient to have a link to the license in the readme

May I know where I can download the df1.csv/df2.csv in the Pandas Operations notebook? Thanks

In the Regression_diagnostics notebook , you are presenting the Shapiro-Wilk test.

The Shapiro-Wilk test's null hypothesis is that the data come from a Gaussian distribution. Therefore, the lower the p-value, the higher the change to reject the Gaussian distribution. The notebook says the opposite:

I don't see any reinforcement machine learning models. Can I add UCB and Thompson sampling? I would love to work on it.

The https://github.com/machine-learning-helpers/docker-python-jupyter project builds a Docker image so that the (your) Jupyter notebooks can be run out-of-the-box on almost any platform in a few minutes.

It gives something like:

• Initialization of the Git repository for the Jupyter notebooks:

$ mkdir -p ~/dev/ml
$ cd ~/dev/ml
$ git clone https://github.com/tirthajyoti/PythonMachineLearning.git

• Initialization of the Docker image to run those Jupyter notebooks:

$ docker pull artificialintelligence/python-jupyter

• Usage:

$ cd ~/dev/ml/PythonMachineLearning
$ docker run -d -p 9000:8888 -v ${PWD}:/notebook -v ${PWD}:/data artificialintelligence/python-jupyter

And then you can open http://localhost:9000 in your browser.

Any modification to the notebooks may be committed to the Git repository (if you are registered as a contributor), and/or submitted as a pull request.

• Shutdown the Docker image

$ docker ps
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS              PORTS                    NAMES
431b12a93ccf        artificialintelligence/python-jupyter   "/bin/sh -c 'jupyt..."   4 minutes ago       Up 4 minutes        0.0.0.0:9000->8888/tcp   friendly_euclid
$ docker kill 431b12a93ccf 

So, all the above could be added to your README.md file.

Hey just wondering, for the How fast are NumPy ops.ipynb

When considering the speed for the log(10) of all the elements in the Numpy array a1, shouldn't you also include the creation of the initial Numpy array?

Line 50 is this:

t1=time.time()
a2=np.log10(a1)
t2 = time.time()
print("With direct NumPy log10 method it took {} seconds".format(t2-t1))
speed.append(t2-t1)

But isn't it more fair to make it this:

t1=time.time()
a1 = np.array(l1)
a2=np.log10(a1)
t2 = time.time()
print("With direct NumPy log10 method it took {} seconds".format(t2-t1))
speed.append(t2-t1)

Considering that it is an additional step not present in the other methods? In your code the bolded line is line 40.

Hi,

I'm wondering what the yesno-fuction does in the following notebook:
https://github.com/tirthajyoti/Machine-Learning-with-Python/blob/master/Regression/Regression_Diagnostics.ipynb

def yes_no(b):
if b:
return 'Yes'
else:
return 'No'

It should decide whether a parameter is significantly important or not for the model?
Where does the b refer to and what's the threshold for it to decide it's not statistically significant?

I usually look at the p-values in the statsmodels-ols table and when they fall below 0.05, they are significant, but in this notebook something else seems to be happening and I'm wondering if you could elaborate a bit on it (What is b?, how is it calculated?, what's the b's threshold? How to change the threshold from 0.01 to 0.05?) When the p-value in the ols-table is above 0.05, but the yes_no-function decides it's significant, what should I do (leave the parameter out or not)?

Kind regards,
Matthias