From #2:

The Tesla stock data isn't in the data folder and the data_reader is broken.

ImmediateDeprecationError: 
Yahoo Actions has been immediately deprecated due to large breaks in the API without the
introduction of a stable replacement. Pull Requests to re-enable these data
connectors are welcome.

See https://github.com/pydata/pandas-datareader/issues

Hi! Following the example in section 13.2.2 to perform logistic regression using sklearn on the acs_ny.csv dataset results in a ConvergenceWarning and doesn't produce an intercept nor coefficients that match those in the book:

import pandas as pd
acs = pd.read_csv('../data/acs_ny.csv')

acs['ge150k'] = pd.cut(acs['FamilyIncome'], [0,150000,acs['FamilyIncome'].max()], labels=[0,1])
acs['ge150k_i'] = acs['ge150k'].astype(int)

predictors = pd.get_dummies(acs[['HouseCosts', 'NumWorkers', 'OwnRent', 'NumBedrooms','FamilyType']], drop_first=True)

from sklearn import linear_model
lr = linear_model.LogisticRegression()

results = lr.fit(X=predictors, y=acs['ge150k_i'])

ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(

To address this warning I increased the max number of iterations as follows:

lr.max_iter = 1000 # my default value was 100

But my intercept and coefficients still don't match those in the book.

Per the books guidance, ran the following commands to get

import numpy as np
values = np.append(results.intercept_, results.coef_)
names = np.append('intercept', predictors.columns)
coefs = pd.DataFrame(values, index = names, columns=['coefs'])
coefs

And these are the results I get:

Very different from those in the book:

I wouldn't be as thrown off if the differences were a few decimal points or so, but my results assign substantially less weight to OwnRent_Outright and to FamilyType_Male Head and I have no idea why...

PS - I'm have VERY little experience with statistics and data science, my background is computer science.

Fifth row from top (as depicted):

"It's just a flesh sound!".find('u') -> Result == 7

Correct:

"It's just a flesh sound!".find('u') -> Result == 6

This may seem minor but newcomers will already be struggling with the zero-reference concept, so this will not be helpful along that journey.

I also am curious about the final line:

"9".zfill(with=5) -> result == '00009'

Having attempted this in both Python 2 and 3, it seems the correct form would simply be:

"9".zfill(5) -> result == '00009'

Hi, I was reading the book and I found a typo in section 1.3.2.2 page 11.

It says iloc and loc will behave om exactly ....

isn't 'om' supposed to be 'in' I can tell they are next to each other on the keyboard.

Cheers

Roque

Greetings!

Kindly refer to chapter 11, point section 11.11 and fig 11.3. If I type the exact code, the plot gives date time data error.

Please suggest a way around.

If I pass a column name to the ebola pandas data frame then it plots for that column. But there are too name columns to type in. But I do not know if this is the best way around.

Regards,
Andy

The text reads-
"This is Python's way of serializing and saving data in a binary format reading pickle data is also backwards compatible."

It supposed to read-
"This is Python's way of serializing and saving data in a binary format. Reading pickle data is also backwards compatible."

In the second to last paragraph, you have in part
"to make sure the axes are apread apart from one another"

I think it should read:
"to make sure the axes are spread apart from one another"

The acs_ny.csv file is missing as well.

Also, has the book version gone to print. There are some typos in the previous couple chapters. I'll be happy to point them out if it is still at a stage that you're able to correct them.

The following code appears on p.49 in 2nd edition.

scientists['age_years'] = (scientists['age_days'].astype('timedelta64[Y]'))

When this is executed in pandas 2.0.3(Python 3.11.5), the following error is output.

Cannot convert from timedelta64[ns] to timedelta64[Y]. Supported resolutions are 's', 'ms', 'us', 'ns

It seems that timedelta64[Y] is no longer supported in pandas 2.0.0 released in April. This seems to be the cause of the error.
https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html#construction-with-datetime64-or-timedelta64-dtype-with-unsupported-resolution

Can the following code be considered as an alternative to the above code?

# Calculate a year in seconds
seconds_in_a_year = 60 * 60 * 24 * 365

# convert timedelta to seconds and then to years
scientists['age_years'] = scientists['age_days'].dt.total_seconds() / seconds_in_a_year

print(scientists)

Please let us know if there is a better way.
Thanks.

In the last non-code paragraph, in the last sentence, you say:

"Notice that if we printed the grouped dataframe, Pandas would return only the memory location."

I believe it does not return the memory location. Rather, it returns the type.

In the fourth to the last line, you have "If we liked, we could manually supply...". I believe the word supposed to be "like" and not "liked"

There is a line on page 27, section 2.2.2 that says "Notice that order is not guaranteed." I think that is no longer true beginning in Python 3.7 as it is a feature of dicts (and technically in 3.6 but an implementation detail). See- https://stackoverflow.com/a/47837132/178550.

Hi Daniel,

Nice course!

This is working well so-far
% conda env create --name pandas_for_everyone -f environment.yml

environment.yml:

name: pandas-data-analysis-chen
channels:
  - anaconda
  - defaults
  - conda-forge
dependencies:
  - python=3
  - pip
  - feather-format
  - ipython
  - jupyterlab
  - numba
  - numpy
  - odo
  - openpyxl
  - pandas
  - pandas-datareader
  - pweave
  - regex
  - scikit-learn
  - seaborn
  - statsmodels
  - wget
  - xlwt

Cheers,
--Peter G

can't find folder tidy-data under data folder, which contains all data files for lesson 7. where can I find these data files ?

In Chapter 6 Data Assembly of the Second Edition,
6.3 Observational Units Across Multiple Tables, (pp.154-160),
the example scripts uses:
data/billboard-by_week/billboard-XX.csv
but I couldn't find the folder (and csv files) in the data section of the repository.

You wrote-
"The Series and DataFrame have a to_csv method to write a CSV file"

I believe is supposed to read-
"Series and DataFrame types have a to_csv method to write to a CSV file"

On section 6.4 "Variables in Both Rows and Columns",
in the last two 'head' prints of "weather" data in page 134,
the day column starts with value 'd1' on the 1st row, but
it jumps with no apparent reason to 'd10' on the 2nd row,
and then it continues with 'd11', 'd12', and 'd13'.
Why jumping? A human error in the editing process?

kneo - the English to Japanese translator of the book.

Hello, I'm a Python Baby and am going through the book line by line.
Section 2.4.1 Boolean Subsetting: DataFrames has a command line which is not performing the way the book suggests.

4 values passed as bool vector

3 rows returned

print(scientists.loc[[True, True, False, True]])

The book says that 3 rows should be returned, index 0, 1 and 3.
Which seems to make sense.

But the Return line says
IndexError: Boolean index has wrong length: 4 instead of 8
Which also seems to make sense, so I'm confused.

Can I get some clarification?
Thanks
Paul

Some of the printed output break the columns over multiple lines

• 2.7.2 Directly change a column
• 6.3.3 code snipit in comment has line break

Hello,

Really appreciating this book. I thought I'd take the time to register that this doesn't add up.

In the pew dataset shown in the below picture, "Don't know/refused" is a value in the religion column:

After we melt with:
pew_long = pd.melt(pew, id_vars="religion")
we should not see this value in the new variable column on page 126. (There is another example of this with different arguments right afterward that also have this value in the income column):

I can't say what went wrong, and I'm on mobile at the moment, but I thought I'd let you know!