kapt-labs / django-check-seo Goto Github PK

Currently, all checks are in a single Class, in the views.py file:

The ideal implementation should be a config parameter with a list of checks files and a Class with a few information (html of the page, content of the page, url, keywords...).

Each check will be in a separate file, with some information available through the class.

Hi!

First of all, I would like to thank you for the great work! This is a very helpful package and It saved us a lot of time on enhancing the SEO of our posts.

Recently, we started using the DJANGO_CHECK_SEO_AUTH to use auth credentials along with the request. But for reasons we don't fully control, the DNS server is redirecting us to another URL.

In this scenario, because of a characteristic of the requests package, the HTTP_AUTHORATION is not sent in the redirect request (after a 3xx) and we end up with a 404 (the view wasn't able to authenticate the request).

I was wondering if the request made by django_check_seo could check for a 3xx
status code and follow the new URL location with the DJANGO_CHECK_SEO_AUTH information.

Here is a simple example:

import requests

r = requests.get(
    URL, 
    auth=(
        settings.DJANGO_CHECK_SEO_AUTH["user"],
        settings.DJANGO_CHECK_SEO_AUTH["pass"],
     ),
    allow_redirects=False
)

if 300 < r.status_code < 400:
    r = requests.get(
    r.headers['location'], 
    auth=(
        settings.DJANGO_CHECK_SEO_AUTH["user"],
        settings.DJANGO_CHECK_SEO_AUTH["pass"],
     ),
)

Versions

• Python 3.6.8
• django-check-seo==0.3.6
• requests==2.18.4

Thank you very much,
[]s

Keywords can be "postgresql", and actual word used in text can be "PostgreSQL".
Check all files to replace soup.element.string to soup.element.string.lower()

I deployed a djangocms in heroku and checked the SEO check but it returned this error HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: //astaqc-djangocms.herokuapp.com//en/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f45e24480b8>: Failed to establish a new connection: [Errno -2] Name or service not known',))

Here is the screenshot

Users may not know where to modify the information.
We could add a button that would open a popup with an instruction manual.

Actual look:

Desired result:
Just add a "guide" or "manual" entry in the CustomList object and add a pop-up in the view:

We don't want to alert about missing alt tags for certain images (like a tracking pixel, or something like this). We could add a list of urls to ignore in the alt tag check.

Since the number of internal links can vary greatly depending on the subject treated, the default limits are "between 1-15" for internal links, and "between 1-5" for external links.

In addition, not having a number of links in these ranges should not trigger a problem, only a warning.

I am not sure whether this will be helpfull to others, but this is what I did after going through some issues while installing this.

You can follow the installation, and after installation make sure this are the settings present.

add 'django.contrib.sites' to INSTALLED_APPS
SITE_ID = 1

This step is important if running without CMS

Go to template 'default.html' in site packages of installed apps
remove 'cms_tag'
save it
access your page seo by,
>. base.domain/django-check-seo/?page=/page_extnsn

it is possible to use without djangocms?
I have a handmade website, and I wanted to see how my SEO is if it is correct or has problems, open some way to install and use this package for sites that are not built from djangocms

If the url contain an accentuated character then the cms populate function will throw an exception.

Currently, all checks are processed in the same way and are displayed in a large list.
A cool thing to add could be a category tag for checks, which could add readability to lists.

The category will be accessible via the Site instance in each check, and it will be of the form "retrieve the list according to its name, or create a new list with the new name".

Old display:

New display:

When using django-check-seo, we can have some logs that says this:

InsecureRequestWarning,
/home/me/projets/projectname/.venv/lib/python3.7/site-packages/urllib3/connectionpool.py:1004:
InsecureRequestWarning: Unverified HTTPS request is being made to host 'my-https-django-website.ext'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings

We should redirect http requests to https ones. And django-check-seo is used in order to analyse public content, that is available all over the web.

Actual way:

• Check if at least one keyword is repeated at least keywords_repeat[0] times.
• Check if all keywords are not repeated more than keywords_repeat[1] times.

New way: check occurrences using percentages of keywords found in total number of words of the content.

• Check if at least one keyword is repeated more than keywords_repeat[0] (0.25%).
• Check if all keywords are not repeated more thant keywords_repeat[1] (1%).

Describe the bug
An <img> missing an alt attribute is filed as Problem.

According to WCAG, some images must not have alt attributes:

Sometimes there is non-text content that really is not meant to be seen or understood by the user. Transparent images used to move text over on a page; an invisible image that is used to track usage statistics; and a swirl in the corner that conveys no information but just fills up a blank space to create an aesthetic effect are all examples of this. Putting alternative text on such items just distracts people using screen readers from the content on the page. Not marking the content in any way, though, leaves users guessing what the non-text content is and what information they may have missed (even though they have not missed anything in reality). This type of non-text content, therefore, is marked or implemented in a way that assistive technologies (AT) will ignore it and not present anything to the user.

Expected behavior
File missing alt attributes as warnings and explain why and how to fill alt attributes.
Quotes and links to WCAG (and the RGAA french equivalent) should be present.

A meta-description is shown for each keyword present in meta keywords:

We want to show only one meta description "searched_in" content if there is only one meta description:

Currently only keywords are slugified when checking presence of keywords in urls.

If the url contain accented characters they will not be slugified.

Ex:

• keyword: keyword éé
• slugified keyword: keyword-ee
• url: https://dom.ext/fr/my-url-which-contain-keyword-éé/

-> slugified keyword is not in url since keyword-ee ≠ keyword-éé

Describe the bug
Error occurs when using an absolute link (/link) during check_links check.

Expected behavior
check_links should append absolute url to domain name.

Bonus
check_links should check if the site.full_url contains env var DOMAIN_NAME to create the domain url (so external tests like http://domain/django-check-seo/?page=https://kapt.mobi should not show any errors).

Adding a config option to allow request module to access protected websites can be useful when:

• a website is protected with a HTTP Authentication system, and django-check-seo is installed on this website:
  the user will fill his/her credentials so the website will become accessible, but django-check-seo will only get the 401 message

When there's no meta description on a page, the check returns "params: only one, found: one, searched in: no data".

Describe the bug
The content inside html conditional comments is not excluded even though it is referenced in DJANGO_CHECK_SEO_EXCLUDE_CONTENT setting.

To Reproduce
Steps to reproduce the behavior:

Considering the following template:

<body>
  <!--[if lt IE 8]>
    <p class="catch-me-if-you-can"></p>
  <![endif]-->
</body>

1. Add DJANGO_CHECK_SEO_EXCLUDE_CONTENT = ".catch-me-if-you-can" in your settings
2. Click on "Check SEO" toolbar button
3. Check the "Raw data" section
4. See that .catch-me-if-you-can HTML tag is not excluded

Expected behavior
.catch-me-if-you-can HTML tag is excluded.

Some users do not know that they can click on a check result to show more informations:

• check result lines have no affordance (no button, no link, no icon)
• mouse cursor when hovering a check result line, is not usual

If we move the Site class out of the views.py file and place it in a checks/ folder (after renaming the checks/ folder containing all control files to checks_list/), we could create a new CustomList class, which will be a new way to organize the textual data of the checks.

new organization:

old way to add a new problem/warning:

new way:

(checks code here)

Describe the bug
When the H1 is in a <nav> tag, django-check-seo does not find it.

<body>
  <nav>
    <h1>not found</h1>
  </nav>
</body>

Describe the bug
Only connected administrators should be able to see the django-check-seo page.

To Reproduce
Grab the url (the thing that looks like http://mysite/django-check-seo/?page=/), and open a private browsing tab.

Expected behavior
The non-connected user should be redirected to a login screen.

Screenshots

bug:

expected behavior:

Currently checks are only searched inside checks/ folder in django-check-seo/ folder (deep inside .venv).

Maybe add a custom path to look for other checks in the project ?

Here's the problem:

https://github.com/kapt-labs/django-check-seo/blob/master/django_check_seo/urls.py#L11

maybe we should check not version.startswith('1') ?

If you compare our old guideline of 100 links and you look at what the web looks like now, it's quite common to have 200 or 300 or 400 links on a page, as long as the page is long, it has value add, there's substantial amount of substance and real stuff on that page.
So the short answer is really not to worry about it, or not to limit yourself to 100 links anymore.
Matt Cutts - https://www.youtube.com/watch?v=QHG6BkmzDEM

The number of links on a page can vary considerably, we just need to make sure that the page has at least one link.

Currently, django-check-seo is searching the content in the <div class="container"></div> tag.

It could be a problem if the main content of the pages of the crawled website is in another tag (like <main> or <div class="cms_main"> ...).

We should provide a setting where we could select a tag/class/list of tags/classes to search content in.

Say you have this keyword:

that's awesome

Then your meta keywords will maybe contain this:

that's awesome

But your html content will contain this:

[...] and that's awesome!

Django check seo does not unescape content in keywords or in meta description (and that's cool because there can be a XSS). However, for websites who escapes specials chars in meta keywords/description tags, maybe we could use a list of authorized chars in the settings, and unescape only the keywords/description tags, like this:

DJANGO_CHECK_SEO_UNESCAPE_AUTHORIZED_CHARS = ["'", "\"", "!", "and", "so", "on", "..."]

I don't really know what to do regarding this issue for now. Its way simpler to just fix the escaping in keywords & description tags.

The goal here is to display data differently than the classic "problem name" - "found X" - "description".

Here's three screenshots, which one display a better interface ?

django-check-seo

audit page in google chrome dev tools

semrush

Sometimes some technical terms are kept in English in the names/descriptions of checks, because their French equivalent does not exist.
It would be better to show that they are written in English.

It seems that all SEO-related websites that speaks about stop words don't have any credible source.

We will not use stop words in the count_words function.

Currently there is not so much informations on what's wrong directly in the "searched in" content:

It seems to be a good idea to show what's wrong (like in this picture; the URL is too deep):

Describe the bug
Failed requests.exceptions.InvalidURL: Failed to parse: http://127.0.0.1:8000b'/fr/'

To Reproduce
Django<3.1
django-cms>=3.7,<3.8
django-check-seo==0.3.6

Expected behavior
Expected http://127.0.0.1:8000/fr/

Desktop :

• Python 3.8.5

** Temporary Fix **
Override IndexView and add at line 30 :

        page = self.request.GET.get("page", None)
        page = page[2:(len(page) - 1)]
        full_url = (
                protocol
                + Site.objects.get_current().domain
                + page
        )

Before:

After:

In-a-perfect-world after:

(because explicit is better than implicit)

Describe the solution you'd like
Currently, keywords are not displayed on the django-check-seo page, and sometimes we need them to check something.

A good improvement could be to display them on the upper part of the page.

Additional context

When we have a page like this:

...
<footer>
  <div class="container">
    Footer
  </div>
</footer>

The self.content var in Site Class only retrieved the <div class="container">...</div> block, and so it is impossible to tell if this item in self.content:

...
<div class="container">
  Footer
</div>

is a footer that need to be removed.

django-check-seo should not include things like header, menu & footer in the core content of the page, but on some websites that's actually the case, leading to the "no keyword found in first §" problem.

If a keyword contain a special char it will not be found in the h1 tag.

It may be helpful to add a mention that the current check is done on the public version of the page (and not the draft).

For example:

7 problèmes trouvés, et 2 avertissements levés sur la page publique

The goal here is to standardizes the way keywords are processed to facilitate the detection of keywords in content.

Ex:

Find keywords in url -> use slugify to effectively compare keywords and url.

Transform something like
"Meta description can be displayed below your page title in search results. If Google find your description too long, it may crop it and your potential visitors will not be able to read all its content. Sometimes, long pertinent meta descriptions will be displayed, but in the vast majority of the results, the description's lengths are 150-170 chars."
to
"The meta description tag can be displayed in search results if it has the right length, and can influence users. And Google classifies sites according to user behaviour."

Ex of links:

Content of the middle empty link:

If content is empty (img with no alt tag), we should display the html tag itself.

Right now, the installation method is to "copy a bunch of files into your project folder, then edit all of them, then spend three in three days debugging this buggy application".

The new approach should be the same as for all other django applications;

• install application using pipenv or pip inside a virtualenv,
• update your settings.py with values,
• add values into your urls.py,
• add a custom file in your project_folder/ to create the button to launch this application.

User currently need to manually install some packages that django-check-seo depends on, or to manually add them to their Pipfile.

But Setup Configuration File syntax include the declaration of dependencies:

[options]
include_package_data = true
packages = find:
install_requires =
    packagename>=version
    otherpackagename

I think the right way to treat dependencies is to let pipenv (or pip) do it for us.

Currently the only factor is the "first called, first ranked" rule of the problems/warnings.

Another thing to do is add an "importance" to the problems/warnings, to allow them to be ordered by importance and remove the "first called, first ranked" rule.

Is your feature request related to a problem? Please describe.
Google does not like dead links on pages. Since checking external links can take time, it is better to test only internal links for now.

Describe the solution you'd like
Solution with requests & request.status_code.

If no check is failed, then no information is displayed.

It could be a good idea to show a list of successful checks.

Old way:

New way:

"trouvé" value should not be "1"