hjwp / book-tdd-web-dev-python Goto Github PK
View Code? Open in Web Editor NEWBook - TDD web dev with Python
Home Page: https://www.obeythetestinggoat.com
License: Other
Book - TDD web dev with Python
Home Page: https://www.obeythetestinggoat.com
License: Other
On page 24 in the pdf version of the book, we have just added a failing unit test (test_bad_maths
). And the step to run the new test is missing relative location of manage.py
.
The line:
$ python3 manage.py test
Should be:
$ python3 ../manage.py test
Hi Harry. Firstly, thanks for writing this book, I am learning a lot.
I went through the Spike for implementing a Persona Login and encountered an error, then went through and did the "real" (test-driven) implentation and encountered the same error.
MultiValueDictKeyError at /accounts/login
""assertion'"
It has something to do with the following code:
user = authenticate(assertion=request.POST["assertion"])
Unfortunately I don't know why this is happening.
When I run the functional tests, I get
selenium.common.exceptions.TimeoutException: Message: ''
While running through this chapter I noticed a small issue that I thought I would flag with the path reference towards the end of the chapter:
https://github.com/hjwp/Book-TDD-Web-Dev-Python/blame/master/chapter_javascript.asciidoc#L737-L738
I believe these should be:
<script src="../static/jquery-3.3.1.min.js"></script>
<script src="../static/list.js"></script>
It's not that important but I just was curious to get the tests running.
First problem was that also cssselect was not installed (so it should be added to tests-requirements.txt), but after that I get this error:
'output:\n', "fatal: '/home/andrea/projects/book_original/source/chapter_11/superlists' does not appear to be a git repository\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n")
I've done a git submodule update --init
but I suspect I need write access to these repos as well is that correct?
In ch11l032
, this is introduced, to test the HTML5 validation:
self.wait_for(lambda: self.browser.find_elements_by_css_selector(
'#id_text:invalid'
))
However, it isn't testing anything.
Because we are using find_elementS_by_css_selector(...)
if the element is not found an empty list will be returned and no exception will be raised. Therefore our wait_for
immediately returns after the first try and we didn't test anything (try making it fail on purpose by looking for a different id, it won't fail).
To fix that, the solution is simple, remove the S
. That way it'll either find that one element, or raise a NoSuchElementException
, which is a WebDriverException
, so our wait_for
works as expected 🙂
In
elspeth@server:$ set -a; source .env; set +a
It would be nice to explain the set -a;
& set +a
. I found out by googling (https://stackoverflow.com/a/43267603/4490991)
In chapter 5 you do the following just to make the test pass and say that is not what we want..
Maybe you should make it more clear to NOT do that because at that point the test is passing, is that what you want?
If you want to have this intermediate step then you can at least say that the functional test is still NOT passing, so the job is clearly not done yet..
from django.http import HttpResponse
from django.shortcuts import render
def home_page(request):
if request.method == 'POST':
return HttpResponse(request.POST['item_text'])
return render(request, 'home.html')
And by the way the following seems a bit excessive to me, it basically uses the same logic that can be used in production.. I know the client will be introduced later but it looks like at this point I have to know already how to implement the solution to actually write a test to prove it..
self.assertIn('A new list item', response.content.decode())
expected_html = render_to_string(
'home.html',
{'new_item_text': 'A new list item'}
)
self.assertEqual(response.content.decode(), expected_html)
Hello, I'm reading early release book and there was something I encounter that's not really your fault but I think it's probably worth mentioning because it took me a couple hours to figure out how to do bug and it could save some people a lot of time.
Early in the book, you show a screenshot of how are your project folder should look and there's a file geckodriver.log in it.
I couldn't figure out how to get this to generate for the life of me until I found one of my other projects that did have it. so i deleted the file, ran it and got a log!
Anyways, it turns out after downgrading selenium the log file generated again so something in their new updates. I guess does not automatically generate logs which might be confusing to some users!
certainly was for me.
cheers
asciidoctor
will warn of the following errors when make chapter_10_production_readiness.html
is ran:
asciidoctor: WARNING: chapter_10_production_readiness.asciidoc: line 374: callout list item index: expected 1, got 3
asciidoctor: WARNING: chapter_10_production_readiness.asciidoc: line 376: callout list item index: expected 2, got 4
When a source block is annotated by callout numbers, same numbers should be used in the annotation text section. But this is not the case in chapter_10_production_readiness.asciidoc
:
.Dockerfile (ch09l005)
====
[source,dockerfile]
----
FROM python:slim
RUN python -m venv /venv
ENV PATH="/venv/bin:$PATH"
COPY requirements.txt requirements.txt <1>
RUN pip install -r requirements.txt <2>
COPY src /src
WORKDIR /src
CMD python manage.py runserver
----
====
<3> We COPY our requirements file in, just like the src folder.
<4> Now instead of just installing Django, we install all our dependencies
by pointing pip at the _requirements.txt_ using the `-r` flag.
Notice the `-r`.
<3> We COPY our requirements file in...
into <1> We COPY our requirements file in...
<4> Now instead of just installing Django...
into <2> Now instead of just installing Django...
When creating the .gitignore file on Windows OS the "" are included in the .gitignore which causes git to not ignore the files.
The following lines in Chapter 1 need to be changed to accomodate Windows users:
From:
$ echo "db.sqlite3" >> .gitignore
$ echo "geckodriver.log" >> .gitignore
$ echo "virtualenv" >> .gitignore
To:
$ echo db.sqlite3 >> .gitignore
$ echo geckodriver.log >> .gitignore
$ echo virtualenv >> .gitignore
When make book.html, Error displayed:
asciidoctor: ERROR: book.asciidoc: line 18: include file not found: /Users/swot/offline/Book-TDD-Web-Dev-Python/part1.harry.asciidoc
asciidoctor: ERROR: book.asciidoc: line 30: include file not found: /Users/swot/offline/Book-TDD-Web-Dev-Python/part2.harry.asciidoc
asciidoctor: ERROR: book.asciidoc: line 42: include file not found: /Users/swot/offline/Book-TDD-Web-Dev-Python/part3.harry.asciidoc
There are typo errors on "Map your domains to the server" chapter in server-quickstart.md file.
The word configration in the sentence "Once you have a domain, you need to set up a couple of A-records in its DNS configration,... "
This is minor but an uppercase is missing in "...you may need to wait. some registrars..."
Instead of importing
from django.core.urlresolvers import reverse
we could import
from django.urls import reverse
Importing from urlresolvers
is only kept for backwards compatibility, and it isn't working
in Django 2.0 anymore.
See: https://docs.djangoproject.com/en/1.11/ref/urlresolvers/
In Chapter 2. Extending Our Functional Test Using the unittest Module the functional_tests.py looks like the following:
from selenium import webdriver
import unittest
class NewVisitorTest(unittest.TestCase): 1
def setUp(self): 3
self.browser = webdriver.Firefox()
def tearDown(self): 3
self.browser.quit()
def test_can_start_a_list_and_retrieve_it_later(self): 2
# Edith has heard about a cool new online to-do app. She goes
# to check out its homepage
self.browser.get('http://localhost:8000')
# She notices the page title and header mention to-do lists
self.assertIn('To-Do', self.browser.title) 4
self.fail('Finish the test!') 5
# She is invited to enter a to-do item straight away
[...rest of comments as before]
if __name__ == '__main__': 6
unittest.main(warnings='ignore') 7
I found that self.browser.quit()
does not close firefox, and instead had to use self.browser.close()
Reading the documentation for selenium, it seems like this is not expected and quit() should have worked.
I am using MacOS Sierra, python3.6, selenium 3.5.0, geckodriver 0.18.0, FF 52.1.0 (64-bit)
Fantastic book so far (thank you!), though just wanted to point out a minor inconsistency with the chapter title and URL numbering that I've just discovered.
The left hand navigational menu item on the obeythetestinggoat.com site for '13. Dipping Our Toes, Very Tentatively, into JavaScript' points to http://www.obeythetestinggoat.com/book/chapter_14.html, whilst '14. Deploying Our New Code' points to: http://www.obeythetestinggoat.com/book/chapter_13.html.
The same behaviour can be seen on the TOC page too (http://www.obeythetestinggoat.com/pages/book.html#toc).
Nothing too serious, tough though you might like to know!
In chapter_10_production_readiness.asciidoc
, when the requirements.txt
is created by adding, there's still an old Django version displayed even though in the next sourcecode
section, the Django version is 4.2.11
:
[subs="specialcharacters,quotes"]
----
$ *pip freeze | grep -i django*
Django==4.2.7 << HERE
$ *pip freeze | grep -i django >> requirements.txt*
$ *pip freeze | grep -i gunicorn >> requirements.txt*
$ *pip freeze | grep -i whitenoise >> requirements.txt*
----
I wanted to take a closer look into the repository for some possible contributions by cloning it to my local machine and followed README for setup.
But I noticed right away that make install
can't be ran without errors:
$ make install
which uv && uv venv .venv || python -m venv .venv
/bin/bash: line 1: python: command not found
make: *** [Makefile:50: .venv/bin] Error 127
Shouldn't python -m venv .venv
be python3 -m venv .venv
? I do have python
set as an alias for python3
in my .bashrc
but it isn't used because the Bash used by make
(SHELL := /bin/bash
defined in Makefile) is a non-interactive shell.
In chapter 5 I'm not sure why you really need database migrations.
So far there were no models and 0 data in the database (and you were using sqlite anyway) so a syncdb would be perfectly enough.
And if you're db is messed up and you're still playing around you can also just remove the .db file..
Also because (how I worked at least) we had only one migration per release, if for any single tiny change there is a different migration it gets very messy and slow..
Maybe you could at least say that migrations are not actually necessary now but as a general practice is good to get used using them?
If you have updated Python to 3.7 version, you will get the following error when trying to do the python3 manage.py runserver
command.
"SyntaxError: Generator expression must be parenthesized"
My code:
def new_list(request):
"""
item.objects.create 是创建Item对象的简化方式无需再掉用.save()方法
"""
form = ItemForm(data=request.POST)
if form.is_valid():
list_ = List()
print(request)
list_.owner = request.user
list_.save()
form.save(for_list=list_)
return redirect(list_)
else:
return render(request, 'home.html', {"form": form})
@patch('lists.models.List')
@patch('lists.forms.ItemForm')
def test_list_owner_is_saved_if_user_is_authenticated(self, mockItemFormClass, mockListClass):
user = User.objects.create(email='[email protected]')
self.client.force_login(user)
#print(settings.AUTHENTICATION_BACKENDS[0])
self.client.post('/lists/new', data={'text': 'new item'})
mock_list = mockListClass.return_value
self.assertEqual(mock_list.owner, user)
I am stucked on chapter with sockets.
I did everything according to the guide:
/etc/nginx/sites-avalible/malomalsky.studio
server {
listen 80;
server_name malomalsky.studio;
location /static/ {
alias /home/malomalsky/sites/malomalsky.studio/static;
}
location / {
proxy_pass http://unix:/tmp/malomalsky.studio.socket;
}
}
../virtualenv/bin/gunicorn --bind unix:/tmp/malomalsky.studio.socket superlists.wsgi:application
Gunicorn works without any errors, but i have 502 Bad request error.
Ubunty 18
The code listings' id-s in Chapter 13 refer to Chapter 11, e.g.: (ch11l018)
When I added another, I chose to just write (ch13l???)
.
In ch07l006, there is an useless assertion that sneaked in.
Basically, that test:
self.assertNotIn('make a fly', page_text)
is useless because in test_multiple_users_can_start_lists_at_different_urls
Edith did not create a second list item with ...make a fly
. So that particular assertion would never fail and just brings a little bit of confusion.
It should be this:
https://hg.mozilla.org/mozilla-central/file/tip/testing/geckodriver
First, the Deasnakes PPA is actually the Deadsnakes PPA, and second, the current link is not working:
https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes
That link gives me an error. I was able to go here:
https://launchpad.net/~deadsnakes
But I am uncertain whether that is the intended destination.
In chapter04 (I tried to comment directly in the code but the asciidoc doesn't let me by the way) there is this:
self.assertTrue(any(row.text) == '1: Buy peacock feathers' for row in rows)
It might be also written like this I think, which would be even easier adding another variable for the text in the rows.
self.assertIn('Buy peacock feathers', [row.text for row in rows])
In chapter 08, "Debugging Tips" part, the error logs of Nginx was written as
/var/log/error.log
But on Ubuntu, the Nginx error log should be
/var/log/nginx/error.log
append(settings_path, '\nfrom .secret_key import SECRET_KEY')
will fail silently bacause of the newline character('\n'). Should change to
run(f"echo '\nfrom .secret_key import SECRET_KEY' >> {settings_path}")
Explains in fabric/issues/1598
Right there, just under The Git version control system, some words ceased to exist. There were not erased, only never pushed.
This is available for any platform, at http://git-scm.com/. On Windows, this comes with the
I would like to add next chapter link at the bottom of this page https://www.obeythetestinggoat.com/book/chapter_unit_test_first_view.html
to make it easier to get to the next page as suggested by the other user. Is it ok that I go ahead?
In Chapter 23: Test Isolation, and “Listening to Your Tests” in the first code snippter after Moving Down to the Forms Layer it says:
class NewListForm(models.Form):
Which was irritating, as there is no models.Form
(PyCharm was complaining). It wasn't a big issue, as it is corrected a few pages later, but it took me a few minutes because I thought I missed something.
There's a mismatch between the prerequisite and preface version listings.
pre-requisite-installations says it was tested on Python 3.12 and might require editing or produce different messages and error logs for other Python versions, whereas the preface notes the book was updated for use with Python 3.11.
The image embedded on the pre-requisite-installations page shows the 3.11 installer as well.
It's a bit confusing when you write:
item.save()
item.full_clean()
while the name of the tests is test_cannot_save_empty_list_item
Because in the end... the item is saved. It can be verified by putting the following right after the with block:
self.assertEqual(Item.objects.count(), 0)
Two potential solutions
Maybe just swapping the order of the 2 lines might make things less confusing :)
item.full_clean()
item.save()
Another solution would be to simply rename the tests from test_cannot_save_empty_list_item
to test_cannot_validate_empty_list_item
And removing the call to save()
alltogether.
In chapter 6.2, in the line
Later on, we’ll refactor out a general wait_for
the later on links to chapter 6.2, but is should link to chapter 11.2
In the online version of the book (Third Edition Early Release), in the section on Installing Django and Selenium, I think that quotation marks might be needed around the Python command when invoking the interpreter with -c
.
Currently, the book suggests this command:
(.venv) $ python -c from selenium import webdriver; webdriver.Firefox()
I had to add quotation marks to make it run:
(.venv) $ python -c "from selenium import webdriver; webdriver.Firefox()"
I tried on Windows 10 (Git Bash and PowerShell), Linux (bash), and Mac (zsh). Quotation marks were needed in each case.
I don't know if quotation marks are strictly necessary in all cases though, as the docs only say that "it is usually advised to quote command in its entirety".
I'm looking forward to the Third Edition! It's great to have access to the early release as it becomes available.
With Python3, the URL is updated to https://diveintopython3.problemsolving.io/.
In chapter 3 one of the test methods is called
def test_home_page_returns_correct_html(self):
this is fine except that it might suggest that you are checking the whole HTML generated, while instead you are checking for the parts you're interested in.
Maybe there could be even two tests:
to make it even more clear
In chapter 5 you use a redirect to re-render the page with the desired post values.
A redirect is usually a more generic way to avoid code duplication, but in this case also just doing this would be enough, and would not have the overhead of a redirect.
def home_page(request):
template_values = {}
if request.method == 'POST':
item_text = request.POST['item_text']
Item.objects.create(text=item_text)
template_values['new_item_text'] = item_text
return render(request, 'home.html', template_values)
There is some question of how to do this in the comments section with a few people unsure of what steps are necessary. One user suggested installing Homebrew and running "brew install geckodriver".
It's useful to know how to place an executable in /usr/local/bin oneself and straightforward to do. Would you like me to make a PR with instructions?
Following the instructions in chapter one you say we can ignore db.sqlite3.
However you never did a syncdb before so the file does not exist unless I missed it, so it might confuse someone to not find a file which should be there..
In the deployment guide after the nginx installation we perform steps to remove the default symlink and set up a proxy pass to django app on port 8000 using a command:
elspeth@server:$ sudo ln -s ../sites-available/$SITENAME /etc/nginx/sites-enabled/$SITENAME
However nginx did not render the relative symlink. My suggestion would be to replace it with:
elspeth@server:$ sudo ln -s etc/nginx/sites-available/$SITENAME /etc/nginx/sites-enabled/$SITENAME
And remove the relative paths in commands in general or add a pwd
before them as it can take to reproduce the desired effect otherwise.
In chapter04 you suggest to commit a change to the functional test (git commit -am 'Functional test now checks..')
Which it's fine, but maybe it suggests people that it's ok to commit broken tests, which should be avoided.
I would commit it only leaving a "skip" maybe or otherwise don't commit it at all and wait for it to pass first..
Another broken link for you, on http://www.obeythetestinggoat.com/book/chapter_philosophy_and_refactoring.html#_on_refactoring
But, as Kent Beck puts it in Test-Driven Development: By Example,
links to http://www.obeythetestinggoat.com/book/chapter_philosophy_and_refactoring.html#tddbe.
import random
chars = 'abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)'
key = ''.join(random.SystemRandom().choice(chars) for _ in range(50))
The above method for secret key generation is good that it's cryptographically secure and fairly strong, but I would advocate for a different method:
import os
import base64
key = base64.b64encode(os.urandom(256 // 8))
The current method is a bit unclear as to how much entropy there is in the key (currently 50 * log2(50) = 282 bits). Drawing random bytes directly from os.urandom
(the same CSPRNG that random.SystemRandom
uses) then base64-encoding them provides a very clear indication of strength: whatever the argument to urandom
is (in bytes, //8
for bits).
Hi,
The section: 'What to do next' -> 'Test for Graceful Degradation' mentions Persona.
At first, I figured I missed something somewhere but after a quick google search I realized it was a leftover from the first edition
Solution could as simple as removing the section.
For this section of the book:
##############################################################################
Switching to Using Unix Sockets
When we want to serve both staging and live, we can’t have both servers trying to use port 8000. We could decide to allocate different ports, but that’s a bit arbitrary, and it would be dangerously easy to get it wrong and start the staging server on the live port, or vice versa.
A better solution is to use Unix domain sockets—they’re like files on disk, but can be used by Nginx and Gunicorn to talk to each other. We’ll put our sockets in /tmp. Let’s change the proxy settings in Nginx:
server: /etc/nginx/sites-available/superlists-staging.ottg.eu
server {
listen 80;
server_name superlists-staging.ottg.eu;
location /static {
alias /home/elspeth/sites/superlists-staging.ottg.eu/static;
}
location / {
proxy_pass http://unix:/tmp/superlists-staging.ottg.eu.socket;
}
}
Now we restart Gunicorn, but this time telling it to listen on a socket instead of on the default port:
elspeth@server:$ sudo systemctl reload nginx
elspeth@server:$ ./virtualenv/bin/gunicorn --bind
unix:/tmp/superlists-staging.ottg.eu.socket superlists.wsgi:application
And again, we rerun the functional test again, to make sure things still pass:
$ STAGING_SERVER=superlists-staging.ottg.eu python manage.py test functional_tests
[...]
OK
Hooray, a change that went without a hitch for once! Moving on.
##############################################################################
I was not so lucky. I was able to create the socket without issue, but the pages wouldn't load. I was seeing the following error logs in nginx:
2018/05/11 16:02:12 [crit] 8944#0: *1 connect() to unix:/tmp/superlists-staging.somesite.com.socket failed (2: No such file or directory) while connecting to upstream, client: 111.111.111.111, se
rver: superlists-staging.somesite.com.socket, request: "GET /favicon.ico HTTP/1.1", upstream: "http://unix:/tmp/superlists-staging.somesite.com.socket:/favicon.ico", host: "superlists-staging.somesite.com", referrer: "http://superlists-staging.somesite.com/"
"No such file or directory" - indicated that nginx couldn't see the socket even though the socket was definitely there:
[nginxuser@otg-staging-server ~]$ ls /tmp/
superlists-staging.somesite.com.socket
systemd-private-xxxxxxxxxxxxxxx-chronyd.service-z1HGQH
systemd-private-xxxxxxxxxxxxxxx-nginx.service-SCaqlr
I found the following on the fedora wiki:
http://fedoraproject.org/wiki/Features/ServicesPrivateTmp
This meant that every service sees a completely different /tmp and can only see its own files in that directory, so nginx was unable to see the gunicorn /tmp files.
To fix it I created a directory called /var/sockets with appropriate permissions and replaced the unix:/tmp/ with unix:/var/sockets/ in both the nginx config file and the gunicorn bind command.
The errors went away and I was able to browse to the website using the new socket configuration.
This is an issue that is probably limited to CentOS 7. (P.S. I had to create a custom policy for selinux as well using the output from "sudo grep nginx /var/log/audit/audit.log | audit2allow -m nginx" following the instructions found here: http://nts.strzibny.name/allowing-nginx-to-use-a-pumaunicorn-unix-socket-with-selinux/)
Chapter http://www.obeythetestinggoat.com/book/chapter_post_and_database.html#_the_django_orm_and_our_first_model:
later on
links to http://www.obeythetestinggoat.com/book/chapter_post_and_database.html#rewrite-model-test which returns to the top of the current page.
[chapter advanced forms]
should display a different string, shouldn't it?
While getting the docker site to work, I ran into an issue where the static files were showing on the docker site and in the functional tests, but not when I ran the site using django's server locally. It turned out that the issue had to do with cached static files. It might be worth mentioning that clearing the cache can solve problems related to static files.
Hey @hjwp,
I hope you are doing well.
I have sent you an email explaining in great detail my issue with FT up until Chapter 7.
I am at Chapter 9 and I cannot deploy now the site due to the current issue.
Can you please check your email and provide me with feedback?
Cheers.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.