The project description should instruct students to ignore the CSV file from version control.

Img link appeared broken in class today

Using functools module to perform memoization in Python (almost as easy as Ruby):

# h/t: http://codingnews.info/post/memoization.html
#  ... https://docs.python.org/3/library/functools.html#functools.lru_cache

from functools import lru_cache

@lru_cache(maxsize=None) # memoizes the results
def test_method():
    print("DOING WORK HERE...")
    time.sleep(10)
    return "COMPLETE"

print(test_method()) # takes 10 seconds
print(test_method()) # returns immediately
print(test_method()) # returns immediately
print(test_method()) # returns immediately

Enhance JSON notes:

import json

with open('data.txt', 'w') as outfile:
    json.dump(data, outfile)

with open('data.txt') as json_file:
    data = json.load(json_file)

https://stackabuse.com/reading-and-writing-json-to-a-file-in-python/

Should add pertinent commands from here into the gspread docs, or maybe a separate doc relating to google credentials which is referenced from the gspread docs:

https://github.com/s2t2/tweet-analyzer-py/blob/master/DEPLOYING.md

heroku buildpacks:set heroku/python
heroku buildpacks:add https://github.com/s2t2/heroku-google-application-credentials-buildpack
heroku config:set GOOGLE_CREDENTIALS="$(< path/to/local/credentials.json)" # references local creds
heroku config:set GOOGLE_APPLICATION_CREDENTIALS="google-credentials.json"

Add links to the notes, with example code:

• https://plotly.com/python/maps/
• https://plotly.com/python/scattermapbox/

import os

import plotly.graph_objects as go
from dotenv import load_dotenv

load_dotenv()

MAPBOX_ACCESS_TOKEN = os.getenv("MAPBOX_ACCESS_TOKEN")

fig = go.Figure(go.Scattermapbox(
        lat=['38.91427','38.91538','38.91458',
             '38.92239','38.93222','38.90842',
             '38.91931','38.93260','38.91368',
             '38.88516','38.921894','38.93206',
             '38.91275'],
        lon=['-77.02827','-77.02013','-77.03155',
             '-77.04227','-77.02854','-77.02419',
             '-77.02518','-77.03304','-77.04509',
             '-76.99656','-77.042438','-77.02821',
             '-77.01239'],
        mode='markers',
        marker=go.scattermapbox.Marker(
            size=9
        ),
        text=["The coffee bar","Bistro Bohem","Black Cat",
             "Snap","Columbia Heights Coffee","Azi's Cafe",
             "Blind Dog Cafe","Le Caprice","Filter",
             "Peregrine","Tryst","The Coupe",
             "Big Bear Cafe"],
    ))

fig.update_layout(
    autosize=True,
    hovermode='closest',
    mapbox=dict(
        accesstoken=MAPBOX_ACCESS_TOKEN,
        bearing=0,
        center=dict(
            lat=38.92,
            lon=-77.07
        ),
        pitch=0,
        zoom=10
    ),
)

fig.show()

Looks like there have been some changes to the gspread package API. https://gspread.readthedocs.io/en/latest/oauth2.html

We should make sure the existing notes still work, and update them as necessary.

gc = gspread.service_account(filename='path/to/the/downloaded/file.json')

UPDATE: new version of gspread with object oriented approach:
https://github.com/prof-rossetti/flask-sheets-template-2023/blob/main/app/spreadsheet_service.py

Student question about when to use python app/my_script.py vs python my_script.py.

Answer is depends on the path structure of the repo where that file is.

Reminder: clarify with students or add to notes.

Add a bite-sized functions exercise like https://github.com/prof-rossetti/georgetown-opim-557-201810/blob/master/exercises/custom-functions.md

Before assigning this project again:

• add requirement that packages should be installed via requirements.txt
• update / clarify README requirements, explicitly mentioning it in the evaluation rubric
• update desired data directory to just be "data" (no nested "monthly-sales" dir)
• for chart, don't need decimals in USD formatting - use whole values
• be more explicit about prompting the user with example valid values, providing instructions to help the user select a file
• use os.path.join and os.path.dirname for reliable filepath construction! ("file should be able to be run from any location on the computer, and still find the file")
• change file name from "dashboard_generator.py" to "top_sellers.py"

Be more explicit in the project description about expectations for the program to instruct the user about the DONE signal.

... to reflect the new os.getenv syntax

Rubric should be more explicit about README requirements mentioned in the Repo setup step. See the robo advisor rubric for a good example.

They've updated installation on Mac. Now can use homebrew (is recommended)

brew tap heroku/brew && brew install heroku

There are some post-installation messages and caveats, but for me it just worked on Zsh.

https://devcenter.heroku.com/articles/heroku-cli#download-and-install

... in the Chart Gallery exercise. Some students were seeing scientific notation to represent the large numbers. And said they were able to solve this with code like:

plt.gcf().axes[0].xaxis.get_major_formatter().set_scientific(False)

Add some example code to the sendgrid notes: h/t: https://github.com/Gplafferty0219/workout-app/blob/master/app/workout.py#L230-L235

import base64
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail, Attachment, FileContent, FileName, FileType, Disposition, ContentId

# ...

def send_email(subject="Workout", html="<p>Today's Workout</p>", pdf="workout.pdf"):
    client = SendGridAPIClient(SENDGRID_API_KEY) #> <class 'sendgrid.sendgrid.SendGridAPIClient>
    message = Mail(from_email=MY_EMAIL, to_emails=MY_EMAIL, subject=subject, html_content=html)
    #attaches the PDF we generated earlier
    file_path = 'workout.pdf'
    with open(file_path, 'rb') as f:
        data = f.read()
        f.close()
    encoded = base64.b64encode(data).decode()
    attachment = Attachment()
    attachment.file_content = FileContent(encoded)
    attachment.file_type = FileType('application/pdf')
    attachment.file_name = FileName('JuicyLiftWorkout.pdf')
    attachment.disposition = Disposition('attachment')
    attachment.content_id = ContentId('Example Content ID')
    message.attachment = attachment
    #send email
    try:
        response = client.send(message)
        return response
    except Exception as e:
        print("OOPS", e.message)
        return None

Next time, ask students to use this template repo, to avoid confusing fork relationships with the upstream repo.

https://github.com/prof-rossetti/codebase-cleanup-template-2021

I think they need to first fork the template, then when creating their own new repo, can choose to do so from their forked template.

After installing Anaconda for the first time on Mac OS Catalina, students might be running into "conda: command not found" related issues.

As a one-time step, locate where the "anaconda3" folder was installed, for example on the desktop, and replace references to conda with ~/Desktop/anaconda3/bin conda.

After initializing the shell for the first time with ~/Desktop/anaconda3/bin conda init zsh(default on Catalina) or ~/Desktop/anaconda3/bin conda init bash, future references to conda should work without need for the prefix.

https://docs.anaconda.com/anaconda/user-guide/troubleshooting/#conda-command-not-found-on-macos-or-linux

doesn't work for a student

Some students were asking for a list of common errors, like what do I do when I see "TypeError: list indices must be integers or slices, not str"

Here are some helpful reference links that could be incorporated into the materials:

• https://www.browserstack.com/guide/learn-software-development-process
• https://www.browserstack.com/guide/types-of-testing
• https://www.browserstack.com/guide/tdd-vs-bdd-vs-atdd

1. Re-structure beginning material to focus on environments, including environment variables. then list all the environments (dev, prod, notebooks, etc), then transition into specific setup of the development environment

2. Can probably lean more heavily on official python tutorial. These links are great resources that I don't believe are currently included in course materials:

   • https://docs.python.org/3/tutorial/datastructures.html
   • https://www.python.org/dev/peps/pep-0202/
   • https://www.python.org/dev/peps/pep-0274/
   • https://realpython.com/list-comprehension-python/

3. More smaller digestible exercises and practice sets

4. Update and extend screen casts

• https://docs.python.org/3/library/pickle.html

import os
import pickle

class Team()
  # ...

FILEPATH = os.path.join(os.path.dirname(__file__), "..", "my", "my-team.pkl")

def save_it():
   team = Team()

    print("SAVING...")
    with open(FILEPATH, "wb") as f:
        pickle.dump(team, f)

def load_it():
    print("LOADING...")
    with open(FILEPATH, "rb") as f:
        team = pickle.load(f)
    return team

FYI: the Team class definition is required for pickle load step, so if defined in separate file, need to import class

Need to tell students to merge their PRs into master branch OR submit the /tree URL pointing to their unmerged branch.

Also prefer to fold in the refactoring and simplification from the beginning, so we can spend more time on projects and less time on stressing out about error-prone Travis integrations. Prefer code climate over Travis next time.

There were no special selections or options. But I did have to do the zsh config. It was weird the conda init code got written into my bash profile, but the shell is zsh. After doing the conda init step things seem ok.

Let's add a challenge to export the images as png files.

Add notes to "/packages/psycopg.md":

import os
from dotenv import load_dotenv
import psycopg2

load_dotenv() #> loads contents of the .env file into the script's environment

DB_NAME = os.getenv("DB_NAME")
DB_USER = os.getenv("DB_USER")
DB_PASSWORD = os.getenv("DB_PASSWORD")
DB_HOST = os.getenv("DB_HOST")

connection = psycopg2.connect(dbname=DB_NAME, user=DB_USER, password=DB_PASSWORD, host=DB_HOST)
print("CONNECTION:", connection)

cursor = connection.cursor()
print("CURSOR:", cursor)

cursor.execute('SELECT * from my_table;')
result = cursor.fetchall()
print("RESULT:", type(result))
print(result)

If you generated your local repo with git init . and it has a "master" branch by default (for example if you're using an old version of local Git), follow these steps to re-configure the local repo:

git branch -m master main
git fetch origin
git branch -u origin/main main
git remote set-head origin -a

Some students experienced issues running python on Git Bash. This is solved by running python -i instead. This caveat should be added to the notes about the python CLI.

https://stackoverflow.com/questions/35471179/python-3-wont-run-from-the-git-bash-command-line

If package installation is required, require usage of requirements.txt file. Students are doing a great job doing this on their robo advisor projects, so they can probably handle it earlier on, and it makes grading much easier 😸

Add notes on time vs space complexity.

References:

• https://stackabuse.com/big-o-notation-and-algorithm-analysis-with-python-examples/
• https://docs.codeclimate.com/docs/cognitive-complexity
• https://docs.codeclimate.com/docs/maintainability

Including how to save files, FTI about "backends", using in notebooks etc.

plt.scatter(_____)


plt.title("_____")
plt.xlabel("______")
plt.ylabel("______")


plt.savefig(img_filepath)

plt.show()

New Notes

References

• https://cloud.google.com/storage/docs/uploading-objects#storage-upload-object-code-sample
• https://cloud.google.com/storage/docs/naming-objects
• https://googleapis.dev/python/storage/latest/client.html
• https://googleapis.dev/python/storage/latest/buckets.html
• https://googleapis.dev/python/storage/latest/blobs.html

Installation

pip install google-cloud-storage

Usage

import os
from pprint import pprint

from google.cloud import storage
from dotenv import load_dotenv

load_dotenv()

GOOGLE_APPLICATION_CREDENTIALS = os.getenv("GOOGLE_APPLICATION_CREDENTIALS", default="google-credentials.json")
GCS_BUCKET_NAME=os.getenv("GCS_BUCKET_NAME", default="my-bucket") # "gs://my-bucket"

if __name__ == "__main__":

    client = storage.Client() # implicit check for GOOGLE_APPLICATION_CREDENTIALS

    print("------------")
    print("BUCKETS:")
    for bucket in client.list_buckets():
        print(bucket)

    print("------------")
    print("BUCKET:")
    bucket = client.bucket(GCS_BUCKET_NAME)
    print(bucket)

    print("------------")
    print("UPLOADING LOCAL FILE:")
    filename = "my_data.csv"
    local_filepath = os.path.join(os.path.dirname(__file__), "..", "data", filename)
    remote_filepath = os.path.join("data", filename)

    blob = bucket.blob(remote_filepath)
    print(blob) #> <Blob: impeachment-analysis-2020, storage/data/mock_graph.gpickle, None>
    print(blob.exists())

    blob.upload_from_filename(local_filepath)
    print(blob) #> <Blob: impeachment-analysis-2020, storage/data/mock_graph.gpickle, 1590433751346995>
    
    print(blob.exists()) #> True

Students are having various issues with getting a working Sendgrid API Key. Posting possible solutions here.

Some students have reported that the default Flask run command doesn't work (could not find app hello.py).

However once we re-organized the app into the "web_app" directory init file and used an app factory function, they reported it started working. In hindsight I think maybe flask was looking for an init file in the root directory, and that's why the hello.py version doesn't work.

Let's update Checkpoint 1 to reflect this student experience, and de-emphasise the need to put all the routes in before we move to the app factory pattern, to make it easier for students to follow along between Checkpoints 1 and 2.

https://github.com/prof-rossetti/intro-to-python/blob/master/exercises/web-app/checkpoints/1-routing.md

conda info
ls -al /opt/anaconda3/envs

list envs:

conda info --envs

• https://docs.sqlalchemy.org/en/13/
• https://docs.sqlalchemy.org/en/13/intro.html
• https://docs.sqlalchemy.org/en/13/dialects/postgresql.html#module-sqlalchemy.dialects.postgresql.psycopg2
• https://docs.sqlalchemy.org/en/13/core/type_basics.html
• https://docs.sqlalchemy.org/en/13/core/metadata.html#creating-and-dropping-database-tables
• https://docs.sqlalchemy.org/en/13/orm/extensions/declarative/api.html
• https://stackoverflow.com/questions/45259764/how-to-create-a-single-table-using-sqlalchemy-declarative-base
• https://www.compose.com/articles/using-postgresql-through-sqlalchemy/
• https://www.learndatasci.com/tutorials/using-databases-python-postgres-sqlalchemy-and-alembic/

# models.py


import os
from dotenv import load_dotenv
from sqlalchemy import create_engine, Column, Integer, BigInteger, String, ARRAY, TIMESTAMP
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm.exc import NoResultFound

load_dotenv()

DATABASE_URL = os.getenv("DATABASE_URL", default="postgresql://username:password@localhost/dbname")
USER_FRIENDS_TABLE_NAME = os.getenv("USER_FRIENDS_TABLE_NAME", default="user_friends")

db = create_engine(DATABASE_URL)
Base = declarative_base()
Base.metadata.bind = db # fixes sqlalchemy.exc.UnboundExecutionError: Table object 'books' is not bound to an Engine or Connection.  Execution can not proceed without a database to execute against.
BoundSession = sessionmaker(bind=db)

class Book(Base):
    __tablename__ = "books"
    id = Column(Integer, primary_key=True)
    title = Column(String(128))
    author = Column(String(128))
    readers = Column(ARRAY(String(128)))

class UserFriend(Base):
    __tablename__ = USER_FRIENDS_TABLE_NAME
    id = Column(Integer, primary_key=True)
    user_id = Column(BigInteger) # , primary_key=True
    screen_name = Column(String(128))
    friend_count = Column(Integer)
    friend_names = Column(ARRAY(String(128)))
    #start_at = Column(TIMESTAMP)
    #end_at = Column(TIMESTAMP)

if __name__ == "__main__":

    #breakpoint()
    #Book.__table__.drop(db)
    #Book.__table__.create(db)
    #if not Book.__table__.exists(): Book.__table__.create(db)
    #if not UserFriend.__table__.exists(): UserFriend.__table__.create(db)
    Base.metadata.create_all(db)

    session = BoundSession()

    try:
        book = session.query(Book).filter(Book.title=="Harry Potter").one()
    except NoResultFound as err:
        book = Book(title="Harry Potter", author="JKR", readers=["John", "Jane", "Sally"])
        session.add(book)
        session.commit()

    print("-------")
    print("BOOKS:")
    books = session.query(Book)
    for book in books:
        print("...", book.id, "|", book.title, "|", book.author, "|", book.readers)


    print("-------")
    print("USER FRIENDS:")
    results = db.execute("SELECT count(DISTINCT screen_name) as row_count FROM user_friends;")
    #print(results.keys()) #> ["row_count"]
    #print(results.rowcount) #> 1
    print("...", results.fetchone()[0]) #TODO: row factory ... results.fetchone()["row_count"]

    session.close()

       self.session = BoundSession()
        batch.append({
            "user_id": row["user_id"],
            "screen_name": row["screen_name"],
            "friend_count": row["friend_count"],
            "friend_names": row["friend_names"]
        })
        session.bulk_insert_mappings(UserFriend, batch)
        session.commit()
        session.close()

• https://docs.github.com/en/github/using-git/setting-your-username-in-git
• https://docs.github.com/en/github/setting-up-and-managing-your-github-user-account/setting-your-commit-email-address
• https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh#step-2-generate-a-new-ssh-key
• https://docs.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent
• https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account

git config --global user.name "Mona Lisa"
git config --global user.email "[email protected]"

git config --global user.name
git config --global user.email

Visit your email settings, you can always choose the noreply email like

git config --global user.email "[email protected]"

Style-checking:

• https://pypi.org/project/autopep8/
• http://pep8online.com/
• https://codeclimate.com/

pip install autopep8

make a commit, then...

autopep8 --in-place --aggressive --recursive .

Add to notes about Env Vars the following excerpt from Twilio SendGrid:

Storing an API key in an environment variable

Twilio SendGrid recommends storing your API key in an environment variable or a config file that is not stored in version control.

When setting a variable in your program, that variable is readable by any person or system that can access the text file where it’s set. However, a variable that's confined to the environment where the code is executed, stored outside the program itself, is called an environment variable. Only people and programs with access to the environment can read the value assigned to an environment variable. This makes environment variables a more secure choice for storing credentials such as API keys.

Once you assign your API key to an environment variable, you can then tell your program to look for and use that variable. Another benefit of this approach is the ability to set API keys with different permissions in different environments such as development, staging, and production without changing the code you deploy to those environments.

Step 1: Fork the "upstream" repo. This creates an "origin" repo under your own control.
Step 2: Clone the forked repo.
Step 3: Add "upstream" remote address
Step 4: Checkout a new feature branch, and make commits there, and push commits to origin branch. Optionally create a PR against your origin repo to do a preliminary code review, post comments, etc.
Step 5. When ready, create a PR against the upstream repo.
Step 6: A maintainer of the upstream repo will review your PR, and if everything looks good, merge your changes in to the upstream master branch.

After your pull request is merged:

git checkout master
git fetch upstream
git pull upstream master
git push origin master

If something goes wrong, you can reset your repo to reflect the current state of the upstream repo:

git checkout master # if not already there
git reset --hard upstream/master
git push origin master

Use the command palette and starty typing "snippets" to find the "Preferences > Configure User Snippets" setting which should yield a snippets JSON file.

See below custom snippet for auto-generating a class constructor method. Todo: placeholder args perhaps.

{
	// Place your global snippets here. Each snippet is defined under a snippet name and has a scope, prefix, body and
	// description. Add comma separated ids of the languages where the snippet is applicable in the scope field. If scope
	// is left empty or omitted, the snippet gets applied to all languages. The prefix is what is
	// used to trigger the snippet and the body will be expanded and inserted. Possible variables are:
	// $1, $2 for tab stops, $0 for the final cursor position, and ${1:label}, ${2:another} for placeholders.
	// Placeholders with the same ids are connected.
	// Example:
	// "Print to console": {
	// 	"scope": "javascript,typescript",
	// 	"prefix": "log",
	// 	"body": [
	// 		"console.log('$1');",
	// 		"$2"
	// 	],
	// 	"description": "Log output to console"
	// }

	"Python Function Definition": {
		"prefix": ["def"],
		"body": [
			"def my_func():",
			"    pass"
		],
		"description": "A function definition in Python."
	},

	"Python Class Constructor": {
		"prefix": ["init", "__init", "defi"],
		"body": [
			"def __init__(self):",
			"    pass"
		],
		"description": "A class initializer method for Python."
	},

	"Python Main Conditional": {
		"scope": "python",
		"prefix": ["main", "__main"],
		"body": [
			"if __name__ == '__main__':",
			"    pass"
		],
		"description": "The main conditional for Python."
	}
}

https://code.visualstudio.com/docs/editor/userdefinedsnippets

See:

• https://github.com/github/renaming
• https://github.blog/changelog/2020-08-26-set-the-default-branch-for-newly-created-repositories/
• https://github.com/settings/repositories

https://realpython.com/instance-class-and-static-methods-demystified/

How to customize CSS and JS? Create a /static directory and then insert home.css and home.js and then load them like:

{% extends "bootstrap_layout.html" %}
{% set active_page = "home" %}

{% block css %}
    <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='home.css') }}">
{% endblock %}

{% block content %}

    <h1>Impeachment Tweet Analysis</h1>

    <p>This is a description.</p>

    <ul>
        <li>First list item</li>
        <li>Second list item</li>
        <li>Third list item</li>
    </ul>

{% endblock %}

{% block js %}
    <script type="text/javascript" src="{{ url_for('static', filename='home.js') }}"></script>
{% endblock %}

console.log("HELLO WURLD... FROM THE HOME PAGE")

console.log(2+2)

h1 {color: royalblue;}

See https://flask.palletsprojects.com/en/1.1.x/tutorial/static/

Some students have asked about grammar / spell-check style red squiggles in their vs code text editor. This is due to pylint not being configured properly. To fix we can just turn off pylint. from the command pallete, "python > enable linting" then "current: off"

aaah that seems to be a local file fix. for a more permanent fix, go to settings:

The monthly sales predictions exercise in unit 5B is a little weak, and should be replaced with something else, like this titanic kaggle competition, which is a lot more fun and instructive:

https://www.kaggle.com/c/titanic
https://www.kaggle.com/c/titanic/data

notes could use a demo screenshot or GIF