https://jenkins.canonical.com/webteam/job/docs.ubuntu.com-deploy-to-staging/123/console

Please add the following web redirects for the MAAS documentation. Thank you!

===

https://docs.ubuntu.com/maas/2.1/en/installconfig-server-iso >>>
https://docs.ubuntu.com/maas/2.1/en/installconfig-iso-install

https://docs.ubuntu.com/maas/2.1/en/installconfig-gui >>>
https://docs.ubuntu.com/maas/2.1/en/installconfig-webui

https://docs.ubuntu.com/maas/2.1/en/installconfig-deploy-nodes >>>
https://docs.ubuntu.com/maas/2.1/en/installconfig-nodes-deploy

https://docs.ubuntu.com/maas/2.1/en/installconfig-kernel >>>
https://docs.ubuntu.com/maas/2.1/en/installconfig-nodes-kernel-boot-options

https://docs.ubuntu.com/maas/2.1/en/installconfig-hwe-kernels >>>
https://docs.ubuntu.com/maas/2.1/en/installconfig-nodes-ubuntu-kernels

The MAAS 1.9 documentation is preserved on docs.ubuntu.com and the API docs are soon to be.

Once #34 is live, it is my plan to redirect all top level sets of documentation to their new home, and also setup specific redirects for pages which have somewhat sensible new homes:

RewriteEngine on

RewriteRule ^/docs/(.*) /docs2.1/$1 [R=301]
RewriteRule ^/docs(2.1|2.0)/releases.html https://launchpad.net/maas/+series [R=301]
RewriteRule ^/docs2.1/changelog.html https://docs.ubuntu.com/maas/2.1/en/2.1-release-notes [R=301]
RewriteRule ^/docs2.0/changelog.html https://docs.ubuntu.com/maas/2.0/en/ref-release-notes [R=301]
RewriteRule ^/docs(2.1|2.0)/https://docs.ubuntu.com/maas/2.0/en/ref-release-notes [R=301]
RewriteRule ^/docs(2.1|2.0)/api.html https://docs.ubuntu.com/maas/$1/en/api [R=301]
RewriteRule ^/docs(2.1|2.0)/api_authentication.html https://docs.ubuntu.com/maas/$1/en/api [R=301]
RewriteRule ^/docs(2.1|2.0)/maascli.html https://docs.ubuntu.com/maas/$1/en/manage-cli [R=301]
RewriteRule ^/docs(2.1|2.0)/version.html https://docs.ubuntu.com/maas/$1/en/manage-cli [R=301]
RewriteRule ^/docs(2.1|2.0)/getting-help.html https://docs.ubuntu.com/maas/$1/en/troubleshoot-getting-help [R=301]
RewriteRule ^/docs(2.1|2.0)/troubleshooting.html https://docs.ubuntu.com/maas/$1/en/troubleshoot-faq [R=301]
RewriteRule ^/docs(2.1|2.0)/man/maas-region.8.html https://docs.ubuntu.com/maas/$1/en/manage-cli [R=301]
RewriteRule ^/docs(2.1|2.0)/man/maas.8.html https://docs.ubuntu.com/maas/$1/en/manage-cli [R=301]
RewriteRule ^/docs(2.1|2.0)/development/philosophy.html https://docs.ubuntu.com/maas/$1/en/intro-concepts [R=301]
RewriteRule ^/docs(2.1|2.0)/hacking.html https://docs.ubuntu.com/maas/$1/en/troubleshoot-faq [R=301]
RewriteRule ^/docs(2.1|2.0)/models.html https://docs.ubuntu.com/maas/$1/en/intro-concepts [R=301]
RewriteRule ^/docs(2.1|2.0)/development/security.html https://docs.ubuntu.com/maas/$1/en/installconfig-gui [R=301]
RewriteRule ^/docs(2.1|2.0)/development/building-packages.html https://docs.ubuntu.com/maas/$1/en/installconfig-package-install [R=301]
RewriteRule ^/docs(2.1|2.0)/development/cluster-registration.html https://docs.ubuntu.com/maas/$1/en/installconfig-rack [R=301]
RewriteRule ^/docs(2.1|2.0)/development/cluster-bootstrap.html https://docs.ubuntu.com/maas/$1/en/installconfig-rack [R=301]
RewriteRule ^/docs(2.1|2.0)/development/tagging.html https://docs.ubuntu.com/maas/$1/en/manage-cli-tags [R=301]
RewriteRule ^/docs(2.1|2.0)/development/preseeds.html https://docs.ubuntu.com/maas/$1/en/installconfig-deploy-nodes [R=301]
RewriteRule ^/docs(2.1|2.0)/development/metadata.html https://docs.ubuntu.com/maas/$1/en/troubleshoot-faq#maas-datasource [R=301]
RewriteRule ^/docs(2.1|2.0)/development/rpc.html https://docs.ubuntu.com/maas/$1/en/installconfig-rack [R=301]
RewriteRule ^/docs(2.1|2.0)/development/notes/* https://docs.ubuntu.com/maas/$1/en/troubleshoot-faq [R=301]
RewriteRule ^/docs(2.1|2.0)/genindex.html https://docs.ubuntu.com/maas/$1/en/ [R=301]
RewriteRule ^/docs(2.1|2.0)/search.html https://www.ubuntu.com/search [R=301]
RewriteRule ^/docs(1.9|2.0|2.1)/(.*) https://docs.ubuntu.com/maas/$1/en/$2 [R=301]

Note that URLs not explicitly mentioned here, e.g. http://maas.ubuntu.com/docs2.1/enum.html, will simply forward through to docs.ubuntu.com and end up on a 404 page (https://docs.ubuntu.com/maas/2.1/en/enum.html). I think this is the correct behaviour for pages where the content simply doesn't exist any more in any form.

@evilnick, @degville, @pmatulis please let me know if these redirects seem to make sense, or if there are others I should add.

@pmatulis requested that this redirect be added:

/maas/2.2/en/installconfig-nodes-hw-testing > /maas/2.2/en/nodes-hw-testing
/maas/2.1/en/installconfig-add-nodes > /maas/2.1/en/nodes-add
/maas/2.2/en/installconfig-add-nodes > /maas/2.2/en/nodes-add
/maas/2.1/en/installconfig-commisson-nodes > /maas/2.1/en/nodes-commission
/maas/2.2/en/installconfig-commisson-nodes > /maas/2.2/en/nodes-commission
/maas/2.2/en/installconfig-nodes-deploy > /maas/2.2/en/nodes-deploy
/maas/2.2/en/installconfig-nodes-power-types > /maas/2.2/en/nodes-power-types
/maas/devel/en/intel-rsd > /maas/devel/en/nodes-comp-hw
/maas/2.2/en/intel-rsd > /maas/2.2/en/nodes-comp-hw

To avoid unnecessary duplication of the data, we should share the built files using the NFS charm.

E.g., follow the owncloud NFS hook implementation.

We should create mirrors of the documentation repositories (currently consumed in rebuild-docs) so they can be automatically pulled from there into the production website anything changes. Then we should configure web-hooks to trigger automatic documentation builds using the hooks set up in #66.

The layout of documentation page doesn't look very tidy.
Here a list of things I would like to be fixed:

1. Searching results page , the top search bar should stay full width (also give 10 px of padding between logo + search bar)
   

2. Menu always positioned on the top-right side of the screen, across the different pages (if the title is too long, break it in two lines)
   

3. Core should be replaced by the original logo: https://assets.ubuntu.com/v1/5051e6e5-core_logo.svg

The three sets of documentation are very confusing:

• docs.ubuntu.com/maas
• maas.ubuntu.com/docs
• docs.maas.io

http://docs.ubuntu.com/maas is the current official documentation. So we should turn off the other old platforms and redirect them here.

This should only be done once #29 is resolved.

The documentation should automatically update when changes are made to a github repository.

https://juju-stats.admin.canonical.com/index/prodstack-comms/ubuntu-docs-ps45/mojo/2017-05-24-autodeploy.log

End:

DISK CRITICAL - free space: / 1 GB (13% inode=62%);| /=8GB;6;7;0;9
2017-05-24 17:56:28 [INFO] 8 OK on webapp/1
2017-05-24 17:56:28 [ERROR] 1 FAIL on webapp/1:
+ /usr/lib/nagios/plugins/check_disk -u GB -w 25% -c 20% -K 5% -p /
DISK CRITICAL - free space: / 1 GB (13% inode=62%);| /=8GB;6;7;0;9
2017-05-24 17:56:28 [WARNING] nagios-check failed (attempt 4 of 6); sleeping for 15s
2017-05-24 17:56:43 [INFO] Retrying (attempt 5 of 6)
2017-05-24 17:56:45 [INFO] 10 OK on haproxy/1
2017-05-24 17:56:47 [INFO] 10 OK on haproxy/0
2017-05-24 17:56:54 [INFO] 8 OK on ubuntu-basenode/0
2017-05-24 17:56:56 [INFO] 8 OK on webapp/0
2017-05-24 17:56:56 [ERROR] 1 FAIL on webapp/0:
+ /usr/lib/nagios/plugins/check_disk -u GB -w 25% -c 20% -K 5% -p /
DISK CRITICAL - free space: / 1 GB (13% inode=62%);| /=8GB;6;7;0;9
2017-05-24 17:56:57 [INFO] 8 OK on webapp/1
2017-05-24 17:56:57 [ERROR] 1 FAIL on webapp/1:
+ /usr/lib/nagios/plugins/check_disk -u GB -w 25% -c 20% -K 5% -p /
DISK CRITICAL - free space: / 1 GB (13% inode=62%);| /=8GB;6;7;0;9
2017-05-24 17:56:57 [WARNING] nagios-check failed (attempt 5 of 6); sleeping for 15s
2017-05-24 17:57:12 [INFO] Retrying (attempt 6 of 6)
2017-05-24 17:57:20 [INFO] 10 OK on haproxy/1
2017-05-24 17:57:21 [INFO] 10 OK on haproxy/0
2017-05-24 17:57:23 [INFO] 8 OK on ubuntu-basenode/0
2017-05-24 17:57:24 [INFO] 8 OK on webapp/0
2017-05-24 17:57:24 [ERROR] 1 FAIL on webapp/0:
+ /usr/lib/nagios/plugins/check_disk -u GB -w 25% -c 20% -K 5% -p /
DISK CRITICAL - free space: / 1 GB (13% inode=62%);| /=8GB;6;7;0;9
2017-05-24 17:57:26 [INFO] 8 OK on webapp/1
2017-05-24 17:57:26 [ERROR] 1 FAIL on webapp/1:
+ /usr/lib/nagios/plugins/check_disk -u GB -w 25% -c 20% -K 5% -p /
DISK CRITICAL - free space: / 1 GB (13% inode=62%);| /=8GB;6;7;0;9
2017-05-24 17:57:26 [ERROR] Nagios checks failed

As previously discussed, both MAAS API docs & Landscape API docs are currently created as ReST files. The simplest way to then include these in this site is to parse those files into Markdown and add them to the existing documentation sets.

It would be nice if this could be an entirely scripted task so it's easy to update them as often as we need. A starting point is (from @degville's work):

pandoc --from=rst --to=markdown+backtick_code_blocks+pipe_tables+definition_lists+compact_definition_lists --column=78 --atx-headers --output output.md input.rst

The above command still has the following problems:

• lists are preceded with a redundant '>' because they're indented in
  the original rst document.
• image path needs to be './media'
• most internal references don't work

I also have a worry about doing this, as also articulated on canonical/ubuntu-core-docs#8, that this breaks the purity of the relationship between the built documentation and the markdown repository. This still needs to be discussed further.

The Dockerfile should be based on the www.ubuntu.com Dockerfile, including:

• Not perform any building of assets - this should be done beforehand
• Copy only the necessary files into the image from this project
• Running the app with python 3 (fixes #64)
• Be based on ubuntu:xenial
• Install any necessary production dependencies into the image
• Take a commit_id argument and create COMMIT_ID env var from it (https://github.com/canonical-websites/www.ubuntu.com/blob/master/Dockerfile#L12)
• Run this site as a webserver on port 80 as its default action

Update the application to be more intelligent about handling /s at the end of URLs.

• Add / to the end of all URLs for index.html pages
• Remove / from the end of all other URLs

Explanation

This issues is to fix the general problem causing issues like #78.

Imagine we have a page, guides/index.html, which lives next to a page, guides/a-guide.html.

In generated documentation pages, all links are relative. So let's assume guides/index.html contains a link <a href="a-guide.html">A guide</a>. You would obviously expect clicking this link to load guides/a-guide.html.

The problem is that as it stands, docs.ubuntu.com will always remove the slash from URLs, so docs.ubuntu.com/core/en/guides/ becomes docs.ubuntu.com/core/en/guides.

Following a link from docs.ubuntu.com/core/en/guides to a-guide will take you to docs.ubuntu.com/core/en/a-guide, instead of docs.ubuntu.com/core/en/guides/a-guide, as it should.

Therefore, rather than simply removing the slash from all URLs, the docs.ubuntu.com app should add slashes to URLs for index pages.

Include documentation from http://maas.ubuntu.com/docs1.9 in this site, as it's still a supported version.

For now, simply copy the site wholesale, styling and all. It's the quickest route.

In either the left or right sidebar, or in the footer, there should be links to file a bug and edit ths page, which take you to github. Is there a way to include the page you came from in the issue description? I think there should be...

From @joanasa89 on March 8, 2017 10:31

Fix search bar & menu position.
Make search bar full width and position menu on top right hand corner (See the attached image)

Copied from original issue: canonical-web-and-design/docs.vanillaframework.io#78

Can you please fix search bar on desktop,medium and small screens.

There should be Google Analytics tracking.

Should be done through documentation-builder with this feature: canonical/documentation-builder#34.

• What parts of the Canonical/Ubuntu ecosystem are/will be talking to the documentation platform?
• Who are the different users across those touchpoints?
• What are their user journeys?
• What is the purpose of the documentation platform for that journey?
• What content will they be accessing, and what format will that be in?

For the Kubernetes deployment system, each set of documentation will be built as a separate Docker image, which can then be deployed at its own subdomain or at a subpath of docs.ubuntu.com.

This means that it no longer makes sense to consider the docs.ubuntu.com app to be the central place that serves all documentation.

So we need:

• A way to serve each set of documentation locally, independently
• A place to keep the Dockerfile for building each set of documentation

These could either live inside each documentation set, or in separate repositories dedicated to the building and releasing of a specific set of documentation, or in this repository. I'm not sure which is best right now, but it needs to be decided.

I am concerned about the main title on docs.ubuntu.com:

Ubuntu documentation

This can cause confusion since official Ubuntu documentation is already hosted on help.ubuntu.com.

d.u.c is like:

"Canonical Ubuntu documentation"
"Canonical Cloud documentation"
"Canonical-sponsored documentation"

It should also include a link to h.u.c :

"For official Ubuntu documentation, go to help.ubuntu.com."

Move https://docs.ubuntu.com to be hosted on Kubernetes, with the new deployment system (as used for https://snapcraft.io).

For the time being, the docs.ubuntu.com application should be move whole-sale, including pulling all sets of documentation into the one application.

High-level goals

• Simplify the process for publishing changes to documentation
  • Currently the publishing process takes at least half an hour and is extremely prone to failure
  • This should be able to reduce that time to 10 minutes or so, and make it much more resilient

Note: Splitting up the documentation sets (out of scope for this epic) will improve the publishing experience further, as each set of documentation can then be published individually.

Background

Instead of the previous plan of allowing this application to auto-update based on github webhooks directly, it now seems like migrating to Kubernetes deployments can make releases so quick and painless that we don't need a smarter system for updates.

I tried to build the docs today but the system failed at the publish-to-staging phase. Send help.

The documentation-builder documentation is pretty basic, but nonetheless all ready in the correct format.

It should included on docs.ubuntu.com.

Otherwise old links to https://docs.ubuntu.com/conjure-up/en/ will continue to 404.

Doc building failed during 'build for staging' phase.
https://jenkins.demo.haus/job/docs.ubuntu.com-code-build/132/console

At a minimum, it should be possible to determine that any local/relative links are valid and emit warnings if they are malformed

Currently redirects are setup manually to redirect e.g. /maas to /maas/2.1/en/.

This should happen automatically - I propose it should always redirect to the top item in the versions file.

There may be a related task to add functionality in documentation-builder / vanilla-docs-theme to notify the user if they're not viewing the latest version of the documentation.

The addition of the stacks doc to docs.ubuntu.com/core sets a precedent for merging several doc sources in a single doc build and is not inline with current usage of the platform. We should have platform-wide pattern and tools for these use cases.

cf. canonical/ubuntu-core-docs#8

I am still worried that it sets a bad precedent. I still think that each set of documentation markdown files should be self-contained, and the relationship between markdown repositories and built HTML files should be direct and simple. And this breaks that.

However, we do need a pattern for drawing together related sets of documentation. This should ideally be codified higher up the chain, with core functionality in documentation-builder or in the docs.ubuntu.com application, rather than hacked together with tools like git-repo in individual documentation sets. But unfortunately it doesn't look like creating and implementing the ideal solution can happen in time for this.

A1. Arrive at the Maas release notes from a search engine or other site.
A2. Try to find the download page.

B1. Arrive at the Ubuntu Core security page from a search engine or other site.
B2. Try to find the download page.

What happens: You can’t get there from here.
What should happen: It’s part of the standard navigation, just like it is on every other part of the project’s site.

One way of fixing this would be integrating the docs as part of their respective project sites. (This would also solve other problems, such as missing/ineffective search.)

Next best would be to let each project embed its own navigation, in toto, onto its section of docs.ubuntu.com. This is apparently the approach taken by docs.docker.com, docs.travis-ci.com, and kubernetes.io/docs: on each, the navigation is nearly consistent with the rest of the site (probably drifting out of sync as one or the other site is tweaked).

The simplest, but least consistent, solution would be to provide a way for each project to link from its docs to its front page, and place that link first in the docs.ubuntu.com header.

To achieve #88, we need to develop a light server application that can serve the built sets of documentation inside each of the documentation sets.

This probably can't be achieved with an off-the-shelf server (like Caddy) as it needs some custom functionality to perform redirects based on special cases:

• Redirect unversioned URLs to the latest version
• Redirect URLs without specified languages to the best available language version
• Redirect to add or remove slashes as appropriate
• Read a redirects.yaml file to set up any custom redirects

This will probably make the most sense as a Flask app, so it gives us the power of Python, but can be contained in a single file that doesn't overly complicate the repository.

The repo command is needed for the updates to the rebuild-docs command (see #42). We should update the README.md to explain how to install it.

I know this issue is being addressed via multiple issues but the Doc team needs a single issue for tracking purposes. It would be great if the aforementioned issues could be cited here.

I will cite the issues that are submitted whenever a publication problem is encountered:

#68
#71
#73
#75

Issue #11 was closed without implementing the bulk of the requirements for the search facility:

• Default technology and software version is based on where the reader initiates the search from
• Include an advanced search dialog that allows for the selection of multiple technologies and software versions (for each)
• Each search result must be clearly associated with a technology and software version. It seems to me that results should somehow be organized by technology and software version
• Make better use of whitespace

Be mindful that documentation may one day be offered in multiple languages. Therefore, the search facility should be ready to accommodate the ability to query by language.

@evilnick
@degville
@matthewhelmke

Each set of documentation should be able to define their own redirects, e.g. in a redirects.yaml file.This file should then be read by this application to redirect any old pages to their new locations.

Ability to search through documentation.

Styling requirements

• Search should be in top right of main nav bar
• Search results display in content area

Before we can implement webhooks, we need to write the methods for pulling and building a set of documentation from the source repository.

Issue

If you check out the project and ./run you have no content; I was told to run ./rebuild-docs to get some content.

This seems to rely on git-extras so I installed git-extras. Unfortunately the released version of git-extras doesn't yet include git-force-clone.

Perhaps consider using a slightly more verbose plain-vanilla git solution to this.

Navigating to https://docs.ubuntu.com/core/en/guides and clicking any of the links on the left hand navigation results in a 404.

Run Python 3 in production. This is a prerequisite to running documentation-builder in the deployed application.

Add an HTTP endpoint for triggering a build of a specific documentation set. To be triggered from the repository host.

For instance, with MAAS, this should work:

https://docs.ubuntu.com/maas/2.1/intro-concepts

instead of having to use

https://docs.ubuntu.com/maas/2.1/en/intro-concepts

The TOC logic strips dots from the hashed links but the ids in the page still have them so they do not anchor.

For example:
http://127.0.0.1:8007/core/en/reference/gadget#the-gadgetyaml-file

Find a way of highlighting syntax in the code blocks.

Try to use the method & colours used in github.

We need the ability for this application to understand and serve several domains. E.g.:

• docs.vanillaframework.io
• docs.snapcraft.io
  etc.

So the app needs updating to read a config file which will map hostnames to set of documentation. E.g.:

docs.vanillaframework.io: templates/vanillaframework/
docs.snapcraft.io: templates/snapcraft/

It should also then redirect from docs.ubuntu.com/{product} to docs.{product}.io in these cases.

This is important specifically for Vanillaframework and Snapcraft, because neither can host their documentation on this platform, as they should, until this is implemented.