imperiumx / python-education Goto Github PK

• About this list
• Learning the language
  • Why learn Python?
  • Beginner
  • Intermediate
• Writing idiomatic Python
• Exercises
  • Code practice websites
  • Small exercises
  • Larger projects
  • My exercises
• Topics
  • Algorithms
  • Best Practices
  • Beyond Python (other programming languages)
  • Boilerplate
  • Celery
  • CLI
  • Code Architecture
  • Concurrency
  • Configuration
  • Debugging
  • Deployment
  • Design patterns
  • Documentation
  • Example, inspiration and template packages
  • Exception
  • File organisation (monorepo, folders, etc.)
  • Functional programming
  • Internals
  • Magic methods
  • Open source Python apps
  • Packages (finding and using them)
  • Packages (opinionated list)
  • Packaging (creating your own package)
  • Parsing
  • Performance
  • Preparing for interviews
  • Quirks and gotchas
  • Regular expressions (regex)
  • Security
  • SQLAlchemy
  • Standard library modules
  • Static analysis of code
  • Tests
  • Tools built with Python
  • Types
  • Unicode
• Reference and other lists
• Staying up to date
• Non-Python professional coding education
• My other lists

Items:

• 🧰 : list of resources
• 📖 : book
• 🎞 : video/movie extract/movie
• 🎤 : slides/presentation
• 🎧 : podcast
• 🔧 : tool
• ⭐️ : must-read

The goal of this documentation is to help you become a productive Python developer.

It assumes that those skills will be used in a professional environment. It includes concrete exercises, because the best way to learn is by doing. It focuses on real-world, applied documentation that will help your programming on a day-to-day basis.

This doc assumes programming knowledge and experience.

If you're also interested in generic programming best practices, I've compiled a list of professional programming resources.

• Why Beginners Should Learn Python
• Why Python keeps growing, explained | The GitHub Blog

Here are some free books:

• Official Tutorial: Python's official doc is really good, highly recommended read.
• Dive Into Python: much faster introduction to Python, a bit outdated but will get you ramped up really fast, especially if you've learned a number of programming language in the past.

The Python Guide has some other good resources.

If you're coming from another language, read this article about the ten myths of enterprise Python.

If you want to review algorithms at the same time, you can use Problem Solving with Algorithms and Data Structures using Python by Bradley N. Miller, David L. Ranum.

Other resources include (prefer the one listed above):

• Automate the boring stuff with Python
• Invent with Python
• Learn Python in one picture
• 11 Beginner Tips for Learning Python Programming
• BeginnersGuide - Python Wiki
• Learning Python — The Hitchhiker's Guide to Python
• Crash into Python: for experienced programmers
• Full Stack Python
• Learn Computer Science with Python, from JetBrains
• How to Use Python: Your First Steps (RealPython)
• Asabeneh/30-Days-Of-Python

I wrote some introductory material to advanced Python:

• Introduction to advanced Python (article).
• Advanced Python presentation (Slideshare).

Some other resources:

• The power of Python descriptors
• dabeaz-course/python-mastery

• 🎞 py-must-watch
• 🎞 calmcode.io

• Muhammad Yasoob Ullah Khalid, Intermediate Python
• Luciano Ramalho, Fluent Python
• Dusty Phillips, Python 3 Object-Oriented Programming
• Brett Slatkin, Effective Python: 59 Specific Ways to Write Better Python

Lists of books:

• Legally Free Python Books List

• Talk Python To Me podcast In-depth interviews and topics.
• Python Bytes Weekly python news and discussion
• Podcast.init Data engineering, data science and DevOps
• Real Python Podcast Python news and education

First things first, let's get code style out of the way. Make sure you've read and memorized PEP8 (code style, more readable version here) and PEP257 (docstring style). Those two code styles are applied by almost all major Python applications and libraries. Use flake8, pydocstyle and black to ensure they are applied (check other linters and autofixers below).

What is called "idiomatic Python" might feel magical at first, especially if you don't know Python. I'd recommend getting your hands dirty with some real world Python code. Try to wander around the code, opening random files. Run the tutorial with a debugger to follow the flow and understand what's going on.

• bottle.py: bottle is a web framework. It's a great resource because it's in all in whole file! Recommended reading. You can even print it.
• flask: another web framework, one of the best. Reading its code is highly recommended as well.

You can find other ideas on this hacker news thread.

I feel it's more important to understand the vision behind Python's design, than to know specific Python idioms. The Zen of Python will help you understand the fundamental reasoning behind each idiom.

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Learning idiomatic Python:

• The Hitchhiker’s Guide to Python
• What the heck does “pythonic” mean?
• Elements of Python style
• Meditations on the Zen of Python
• 3 Tips For Writing Pythonic Code

List of books:

• Awesome Python Books

The best way to learn is to do.

• Exercism
• Codingame
• Solve some of the Project Euler problems
• LeetCode
• Codewars
• HackerRank

• Create a virtual environment with virtualenv.
• Fix a bug in one of the Python packages listed in the Python guide.
• Take inspiration from this list of Raspberry Pi projects on reddit

• Build a lock library for Redis.
• Build a cache library for Redis.
• Build an API for storing todos
• Build an API for next bus departure time.
• Build an ORM for a SQL database.
• Build a command line parser.
• Build a template engine.
• Build a static site generator.
• Build an HTTP library.
• Clone one of those games
• Write a game using pyarcade
• spandanb/learndb-py: learn database internals by implementing it from scratch.

Reddit's dailyprogrammer subreddit has some good exercises as well.

Another great way to learn Python is by contributing to one of the numerous open source libraries. Coding is not something that is to be learn in isolation, and you'll learn great valuable insights from the code review you'll get from those communities. You can look for tickets (e.g. Github issues) for Python libraries you're using, or find some in the list below, or pick one of the libraries listed in Awesome Python.

Make sure you pick a library where the tickets are not too involved, and where the community is still alive (i.e. there's recent merged pull requests).

• Check intro-to-python/exercises.
• Check advanced-python/exercises.

• TheAlgorithms/Python: all algorithms implemented in Python

• The Best of the Best Practices (BOBP) Guide for Python
• When Python Practices Go Wrong: a pretty opinionated presentation that can be too concise at times, but nonetheless very interesting for somebody looking to constrain their creativity with Python constructs.
• Stop naming your python modules "utils"
• zedr/clean-code-python: Clean Code concepts adapted for Python
• Django for Startup Founders: A better software architecture for SaaS startups and consumer apps: I strongly disagree with some of the points, but there's definitely some inspiration to take.
  • Keep business logic in services
  • Make services the locus of reusability
  • Use functions, not classes
  • There are exactly 4 types of errors
  • Use serializers responsibly, or not at all
  • Write admin functionality as API endpoints
  • Keep logic out of the front end

• 27 languages to improve your Python
• List of languages that compile to Python
• Rust for Python Programmers, Armin Ronacher
• What learning APL taught me about Python

• An opinionated Python boilerplate
  • pip-tools
  • Makefile
  • ruff
• A Python project checklist

Celery is a distributed async tasks runner.

• Celery best practices, Balthazar

Building command line interfaces.

• Building an authenticated Python CLI

• The clean architecture
• python-clean-architecture
• 📖 Cosmic Python: simple patterns for building complex Python application (free)
  • Domain modeling and DDD
  • Repository, Service Layer, and Unit of Work patterns
  • Event-driven architecture
  • Command-query responsibility segregation
• Arc Note: Datasette

• Concurrency with Python: a pretty complete series of articles that goes into threads, functional programming, actor models, CSP, coroutines and data intensive architectures.

• Best Practices for Working with Configuration in Python Applications
  • Use identifiers rather than string keys to access configuration values.
  • Use static typing
  • Validate early.
  • Declare close to where it is used.
• Doing Python Configuration Right
  • Static things that don't change often, or things that dramatically influence the behavior of the system should live in the code.
  • Dynamic things that change frequently, or things that should be kept secret (API keys/credentials) should live outside the code.
• Use TOML for .env files?

• General introduction: Python debugging tools
• A better debugger: pudb
• Debugging Python Like a Boss
• Syntax Error #11: Debugging Python

See also the more generic Docker section in charlax/professional-programming.

• Production-ready Docker packaging for Python developers

• Python 3 Patterns, Recipes and Idioms
• Decorators in 12 steps
• Design pattern templates in Python (Github)
• Python Design Patterns Guide: a nice intro to design patterns in Python
• faif/python-patterns: a collection of design patterns and idioms in Python.
• Python Design Patterns
  • The Composition Over Inheritance Principle
• Design Patterns in Machine Learning Code and Systems

I maintain a list of antipatterns on this repo.

• pandas is a great example to follow (using Sphinx, separating into quickstart, user guide, API reference).
• Example NumPy Style Python Docstrings
• Why you should still read the docs

• charlax/cookiecutter-python-api: a cookiecutter template for an HTTP API with lots of best practices: mypy, flake8, isort, black, Makefile, fastapi, DDD pattern, file organization, etc.

By topics:

• Authorization: holinnn/deny

• The Most Diabolical Python Antipattern

• OpenDoor's Python Monorepo
• Atlas: Our journey from a Python monolith to a managed platform (Dropbox)
• From Chaos to Cohesion: Architecting Your Own Monorepo

Consider checking out the same section on my charlax/professional-programming repo.

• Python Partials are Fun!
• sfermigier/awesome-functional-python
• Functools - The Power of Higher-Order Functions in Python
• Functors, Applicatives, And Monads In Pictures
• Monads in 15 minutes
• more-itertools/more-itertools: more routines for operating on iterables, beyond itertools
• jmesyou/functional-programming-jargon.py: jargon from the functional programming world in simple terms

• Why Python is Slow: Looking Under the Hood
• The internals of Python string interning
• Python Data structures
• Python’s Innards: Introduction
• 15. Floating Point Arithmetic: Issues and Limitations

Python behind the scenes series:

• Python behind the scenes #11: how the Python import system works
• Python behind the scenes #10: how Python dictionaries work
• Python behind the scenes #13: the GIL and its effects on Python multithreading

• A Guide to Python's Magic Methods

It's often a good idea to read the Python source code of well-written applications:

• mahmoud/awesome-python-applications: free software that works great, and also happens to be open-source Python

How to use:

• setup.py vs. requirements.txt: this is an important gotcha for any library developer.
• Overview of python dependency management tools
• Virtual Environments Demystified
• Commit your poetry.lock file to version control

Lists:

• Awesome Python provides a great list of third party libraries.
• ml-tooling/best-of-web-python: awesome Python libraries for web development

Here's a short list of great packages:

• 🔧 willmcgugan/rich: rich text and beautiful formatting in the terminal
• 🔧 tqdm: wrap any iterable and show a smart progress meter
• 🔧 tomerfiliba/plumbum: shell combinators

Some other cool packages:

• Acreom/quickadd: parse natural language time and date expressions

• Packaging guide
• Open Sourcing a Python Project the Right Way
• Tips for your Makefile with Python
  • Environment variables with default
  • Full Example with Poetry
• 🔧 commitizen-tools/commitizen: create committing rules for projects, auto bump versions and auto changelog generation
• 🔧 nedbat/scriv: changelog management tool
• Makefile tricks for Python projects

• Learn Python ASTs, by building your own linter

• ⭐️ Process large datasets without running out of memory
  • Measuring memory usage in Python: it’s tricky!
• A guide to analyzing Python performance
• cProfile module documentation
• Using qcachegrind to visualize profiling data
• How vectorization speeds up your Python code
• You Should Compile Your Python And Here’s Why
• When Python can’t thread: a deep-dive into the GIL’s impact, PythonSpeed
  • When does a Python thread need to hold the GIL?
  • The parallelism implications of the GIL
  • The good scenario: Long-running C APIs that release the GIL
  • Bad scenario #1: “pure” Python code
  • Bad scenario #2: Long-running C/Rust APIs, but author forgot to release GIL
  • Bad scenario #3: Low-level code with pervasive Python C API usage
• CI for performance: Reliable benchmarking in noisy environments, PythonSpeed
  • Problem #1: Inconsistent results on a single machine
  • Problem #2: Inconsistent results across machines
  • Use Cachegrind

Stories:

• Profiling Python using cProfile: a concrete case
• Example cProfile session
• How we optimized Python API server code 100x
• An optimization story

Tools:

• 🔧 SnakeViz is a browser based graphical viewer for the output of Python’s cProfile module.
• 🔧 RunSnakeRun is a small GUI utility that allows you to view (Python) cProfile or Profile profiler dumps in a sortable GUI view.
• 🔧 plasma-umass/scalene: a high-performance, high-precision CPU, GPU, and memory profiler
• 🔧 pytest-benchmark
• 🔧 benfred/py-spy: sampling CPU profiler written in Rust for low overhead
• 🔧 pythonspeed/filprofiler: memory profiler for data processing and scientific computing applications

• donnemartin/interactive-coding-challenges: 120+ interactive Python coding interview challenges (algorithms and data structures). Includes Anki flashcards.

• Hidden features of Python
• 30 Python Language Features and Tricks You May Not Know About
• A collection of Python "wat" moments
• satwikkansal/wtfpython: a collection of interesting, subtle, and tricky Python snippets
• Ned Batchelder, Facts and myths about Python names and values
• Floating Point Arithmetic: Issues and Limitations, Python Docs

• Interactive regex exercises
• The unreasonable effectiveness of f-strings and re.VERBOSE

• Never Run ‘python’ In Your Downloads Folder
• ossillate-inc/packj: The vetting tool 🚀 behind our large-scale security analysis platform to detect malicious/risky open-source packages

SQLAlchemy is the de facto standard ORM for Python. It has a unique approach: contrary to most ORM, it tries very hard not to hide the SQL implementation details from you. This is great because it forces you to really understand the underlying DB.

Here is some slightly outdated content that is super useful to fully leverage the library:

• Watch the SQLAlchemy introduction video. It lasts 3 hours but is extremely insightful, and introduces to some great object oriented patterns.
• Handcoded application with SQLAlchemy
• Alembic is a lightweight database migration tool for usage with the SQLAlchemy Database Toolkit for Python.

• Python built-in functions to know
• Understanding all of Python, through its builtins

Some little known standard library modules:

• shelve: Python object persistence

• Essential python tools - Quality

Half of coding time is usually spent writing tests. Yet how to write tests efficiently is very rarely taught at school - even though it came make a huge difference in engineering productivity and quality.

First, make sure you're familiar with the different kind of testing strategies laid out in Testing Strategies in a Microservices Architecture (Martin Fowler).

Then, read some of those articles:

• Mock yourself, not your tests: great articles about the danger of mocking, and better unit testing strategies.
• Building Good Tests, Chris NeJame
  • 1 assert per test function/method and nothing else
  • Use standard assert statements, instead of the unittest.TestCase assert methods
  • Test behavior, not implementation
  • Only verify state-changing method calls
  • Test the result, not the process
  • Every test should be able to be run in parallel with any other test
  • A test should never be flaky
  • Try to avoid mocking things whenever possible.
  • Test coverage is not a metric for what was tested; it’s a metric for what code your tests managed to hit
  • The code should be easy to test * Make your test code succinct and idiomatic
• pytest is a test framework. It's very elegant and allows to quickly write very maintainable tests.
• My Python testing style guide
  • Assert results and outcome, not the steps needed to get there
  • Use real objects for collaborators whenever possible
  • A mock must always have a spec
  • Consider using a stub or fake
  • Consider using a spy
  • Don't give mock/stubs/fakes special names
  • Use factory helpers to create complex collaborators
  • Use fixtures sparingly

• pz: easily handle day to day CLI operation via Python instead of regular Bash programs.

• The state of type hints in Python: a good summary of typing in Python and its gotchas.
• Static Typing Python Decorators
• Static Duck Typing in Python with Protocols
• Writing Python like it’s Rust

• Solving Unicode Problems in Python 2.7
• Unicode Howto in Python 3 (official Python documentation).
• The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

• Best Python Resources
• Awesome Python
• PyCon 2015
• Invent With Python, Further Reading: Intermediate Python Resources

There are two main newsletters for Python, which mostly cover the same things:

• Pycoder's Weekly
• Python Weekly

You can also checkout the Python subreddit.

Read this up on my professional programming doc.

If you want to get in touch, checkout my website.

• engineering-management
• entrepreneurship-resources
• professional-programming
• python-education