Code Monkey home page Code Monkey logo

pytest-copie's Introduction

pytest-copie

License: MIT conventional commit Black badge prettier badge pre-commit PyPI version Conda build Test Coverage Documentation Status

Overview

pytest-copie is a pytest plugin that comes with a copie fixture which is a wrapper on top the copier API for generating projects. It helps you verify that your template is working as expected and takes care of cleaning up after running the tests. πŸ“’

It is an adaptation of the pytest-cookies plugin for copier templates.

It’s here to help templates designers to check that everything works as expected on the generated files including (but not limited to):

  • linting operations
  • testing operations
  • packaging operations
  • documentation operations
  • …

Installation

pytest-copie is available on PyPI and can be installed with pip:

pip install pytest-copie

Usage

The copie fixture will allow you to copy a template and run tests against it. It will also clean up the generated project after the tests have been run.

def test_template(copie):
    res = copie.copy(extra_answers={"repo_name": "helloworld"})

    assert res.exit_code == 0
    assert res.exception is None
    assert result.project_dir.is_dir()

Context and template location can be fully customized, see our documentation for more details.

Credits

This package was created with Copier and the @12rambau/pypackage 0.1.11 project template.

pytest-copie's People

Contributors

12rambau avatar davidbrownell avatar dependabot[bot] avatar genevievebuckley avatar tschm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

pytest-copie's Issues

New release (v0.1.3)

Is your feature request related to a problem? Please describe.
I am using pytest-copie in a projec and our CI jobs are failing because of errors happening in the test teardown process (details here).

Describe the solution you'd like
I would like there to be a new release of pytest-copie.
Then we could pin the version in our CI environment, and avoid the failing tests.

Recent PRs #51 and #56 are very useful, and I think it would be helpful to make them widely available.

Describe alternatives you've considered
We can also install pytest-copie on our CI from the main branch on github, eg: pip install git+https://github.com/12rambau/pytest-copie.git. That would work well, and then later on we could pin to the next release whenever it becomes available.

Usage docs page example does not work correctly

Describe the bug
The example demo from the Usage docs page does not work. Multiple errors are produced.

To Reproduce
Steps to reproduce the behavior:

  1. Create a new test environment, with python 3.10, pip, copier, pytest, and pytest-copie.
  2. Go to the Usage docs page
  3. Create this repository structure:
my_template/
β”œβ”€β”€ {{repo_name}}/
β”‚   └── README.rst.jinja
β”œβ”€β”€ tests/
β”‚   └── test_template.py
└── copier.yaml
  1. Add this content to the file copier.yaml
repo_name:
   type: str
   default: foobar
short_description:
   type: str
   default: Test Project
  1. Add this content to the file README.rst.jinja:
{{ repo_name }}
===============

{{ short_description }}
  1. Add this content to the file ``:
def test_template(copie):
    result = copie.copy()

    assert v.exit_code == 0
    assert result.exception is None
    assert result.project_dir.name == "foobar"
    assert result.project_dir.is_dir()
  1. Confirm that regular old copier works on this example repo: cd ..; copier copy my_template my_output. Observe that it works, and has created a "foobar" directory containing a README.rst file containing a "foobar" heading and the example short description.
  2. Try running pytest (with pytest-copie on the my_template repository: cd my_template; pytest --keep-copied-projects --pdb
  3. See the pytest errors

Additional context
There were a number of errors. I've found and made fixes for some easy ones, but am still not quite sure about some others.

  • Easy fix number one: #46
  • Easy fix number two: #47
  • Remaining problem:
    • ...this sounds like it has an easy fix, i.e. adding _subdirectory: {{repo_name}} to copier.yaml
    • BUT that causes other problems. I tried:
      • _subdirectory: {{repo_name}}
      • _subdirectory: "{{repo_name}}"
      • _subdirectory: \{\{repo_name\}\}
      • _subdirectory: my_template
      • _subdirectory: foobar (I know this one is the silliest, but I was getting fed up)
    • A workaround that helped was to restructure the repository, so we had my_template/template/{{repo_name}}/README.rst.jinja, and in copier.yaml I wrote _subdirectory: template. That worked, and got me to the next error.
  • Possibly related problem - pytest-copie's result.project_dir.name is returning copie0001 instead of the expected foobar. This is possibly related to the subdirectory problem above, because if you look in the temporary pytest-copie folder that is generated, the foobar directory (and everything else we expect) does exist... it's just inside the copie0001 temp folder. So if we can convince pytest-copie to look one level down for the project_dir, that might fix this.

Add copie.update functionality.

Is your feature request related to a problem? Please describe.
One of the big motivations to move from cookiecutter to copier is the ability to run the copier update command. It would be nice to be able to test it.

Describe the solution you'd like
I would probably just start simple where we could run update changing one of the data values to see that the template updates to replicate this behavior:
copier update --defaults --data updated_question="my new answer"

The code would likely look like:

def test_template(copie):
    result = copie.copy()
    result = copie.update(result, extra_answers={"updated_question": "my new answer"})
    # asserts go here...

A more advanced solution would allow us to save on old version of the template as a subdirectory in the test folder, then we could pass it as a copie Result pytest fixture, then run the copie.update() against it.

The code could look something like:

@pytest.fixture
def old_template(copie):
    return copie.template('./tests/old_template')
    
def test_template(copie, old_template):
    result = copie.update(old_template)
    # asserts go here...  

Add copie_session fixture

Is your feature request related to a problem? Please describe.
As part of switching our current test suite from pytest-cookies to pytest-copie, we were held up by the lack of a copie equivalent for the cookies_session fixture. This meant we got a lot of ScopeMismatch errors.

Describe the solution you'd like
A copie_session fixture that is session-scoped.

answers are not processed if they use template

using the following default copier.yml:

# questions
project_name:
  typr: str
  default: "Package skeleton"
  help: "The name of the project"

github_repo_name:
  type: str
  default: "{{ project_name.lower().replace(' ', '-') }}"
  help: "The name of the GitHub repository"

project_slug:
  type: str
  default: "{{ github_repo_name.lower().replace('-', '_') }}"
  help: "The slug that will be used to name the lib package"

will not replace the values in the default. making them impossible to use.

!include config does not work

Describe the bug
Trying to run the test on a project with split configuration does not work.

A simple test produces the following error:

 <Result expected a single document in the stream\n  in "<unicode string>", line 2, column 1:\n    !include src/includes/shared.yml\n    ^\nbut found another document\n  in "<unicode string>", line 3, column 1:\n    ---\n    ^>.exit_code

To Reproduce
at the start of your copier.yml add:

---
!include example.yml
---

then create a new example.yml with a question

Run pytest

test_file.py

def test_template_with_extra_answers(copie):
    result = copie.copy()
    assert result.exit_code == 0

Conflicts with pytest-cookies

This is just a note, but I found this library because I'm a fan of the pytest-cookies library and this one seems to follow the same strategy. That said, my first attempt at using this I ran into argparse.ArgumentError: argument --template: conflicting option string: --template error. Having both pytest-cookies and pytest-copie installed led to this error.

The obvious fix is to uninstall pytest-cookies, and then to just keep them separated by different virtual environments.

I don't think there's much to do to fix it, but I wanted to create this issue because it's a cryptic error and others will likely have trouble figuring it out. Two, I think it would benefit to add a warning note somewhere in the documentation or readme.

Support different fixtures for scopes

The tmp_path fixture has scope "function". Hence your copie fixture will always have the same scope which makes it less useful than it could be.

I have worked first on a fixture for creating a temp_path (for dst) that has a larger scope, e.g

@pytest.fixture(scope="session")
def copier_dst(tmp_path_factory):
    target = tmp_path_factory.mktemp("copier")
    LOGGER.info(target)
    return target

and then doing the expensive construction of the template only once for all tests...

@pytest.fixture(scope="module")
def result(src_path, copier_dst):
    run_copy(
        src_path=src_path,
        dst_path=copier_dst,
        unsafe=True,
        defaults=True,
        user_defaults={"project_name": "maffay"},
    )

    os.chdir(copier_dst)

    # fire off poetry
    os.system("poetry update -vv")
    # yield the folder
    yield copier_dst
    # the path is deleted at the very end when all tests have finished
    shutil.rmtree(copier_dst)

I am now exposing the built template and can run various tests against it.

What do you think? Please share your thoughts.

clean pyproject.toml

remove libs that are not used in the new documentation (btn, icon, design etc...)

Update template to 0.1.11

The package is based on the 0.1.5 version of @12rambau/pypackage.

The latest version of the template is 0.1.11.

Please consider updating the template to the latest version to include all the latest developments.

Run the following code in your project directory to update the template:

copier update --trust --defaults --vcs-ref 0.1.11

Note
You may need to reinstall copier and jinja2-time if they are not available in your environment.

After solving the merging issues you can push back the changes to your main branch.

Testing Validators

Note sure if I'm doing something wrong or if pytest-copie isn't working.

I have a simple copier.yml with a validator:

project_name:
    type: str # Any value will be treated raw as a string
    help: An awesome project needs an awesome name. Tell me yours.
    default: paradox-specifier
    validator: >-
        {% if not (project_name | regex_search('^[a-z][a-z0-9\-]+$')) %}
        project_name must start with a letter, followed one or more letters, digits or dashes all lowercase.
        {% endif %}

Running copier copy -d project_name=Any$invalid_answer! . template_copy throws an expected ValueError, but I'm not able to replicate this behavior in pytest-copie. Here's my test.

def test_name_validator(copie):
    result = copie.copy(extra_answers={'project_name': '@nything-You_Want$'})

    assert result.exit_code != 0
    assert result.exception is not None

Please and thank you!

pytest-copie does not recognise copier.yml, only copier.yaml

Describe the bug
The copier library accepts as input configuration files copier.yml or copier.yaml. However, pytest-copie only recognises copier.yaml, because it is hard-coded on this line:

copier_yaml = template_dir / "copier.yaml"

I think if copier accepts this as input, pytest-copie should do too.

To Reproduce
Steps to reproduce the behavior:

  1. Take any currently working repository that pytest-copie can run on
  2. Change the file extension of the config from .yaml to .yml, eg: mv copier.yaml copier.yml
  3. Run pytest and see the error caused by pytest-copie

display build errors in the pytest report

currently, when somthing goes wrong while building the template, the errors are saved in the result.exception variable which makes it dificult to understand what is going wrong with the template code. I think I should raise errors to let pytest display them in the final report by default.

Pytest-copie don't use the HEAD version if vcs-ref exist

I have a templlate that is introducing a bug in the resolution and I realized that the test are still run on this published version instead of the one stored in the local files.

It's related to this issue: copier-org/copier#1484 in the main copier but I don't manage to enforce/test this behaviour here.

I thought naively that #70 was sufficient but it's not.

Graceful error handling in pytest-copie teardown process

Describe the bug
Over at the napari-plugin-template repo we've been struggling to make our CI pass with pytest-copie.

I've found that if there are any errors in the pytest-copie teardown process here, then pytest reports errors, failing the build. Would it be possible to either set shutil.rmtree(test_dir, ignore_errors=True), or have ignoring errors at this point be a user configuration option we can set somehow?

Additional details
The reason shutil.rmtree is producing errors for us is that there are some write protected files being generated. We don't really care if the teardown process leaves some files intact - for us it is much more important that the github CI runner shows us whether the tests pass or fail.

(We are ending up with write protected files because our template is running a task to run git init and git add + git commit on the newly created repository. Git is producing some write protected files (weirdly, this is only a problem for us on Windows) and I haven't been able to figure out if there's a way to modify the permissions. There seems like it should be possible to do, but the immutability of git commits is making things difficult)

To Reproduce

I have a minimal, reproducible example on the tox-git-teardown-problem branch of this repository: https://github.com/GenevieveBuckley/demo_template/tree/tox-git-teardown-problem

(pytest-copie-dev) C:\Users\admin\Documents\GitHub\demo_template>pytest
================================================= test session starts =================================================
platform win32 -- Python 3.10.13, pytest-7.4.3, pluggy-1.3.0
rootdir: C:\Users\admin\Documents\GitHub\demo_template
plugins: copie-0.1.2
collected 1 item

tests\test_template.py .E                                                                                        [100%]

======================================================= ERRORS ========================================================
_________________________________________ ERROR at teardown of test_template __________________________________________

request = <SubRequest 'copie' for <Function test_template>>
tmp_path = WindowsPath('C:/Users/admin/AppData/Local/Temp/pytest-of-admin/pytest-19/test_template0')
_copier_config_file = WindowsPath('C:/Users/admin/AppData/Local/Temp/pytest-of-admin/pytest-19/user_dir0/config')

    @pytest.fixture
    def copie(request, tmp_path: Path, _copier_config_file: Path) -> Generator:
        """Yield an instance of the :py:class:`Copie <pytest_copie.plugin.Copie>` helper class.

        The class can then be used to generate a project from a template.

        Args:
            request: the pytest request object
            tmp_path: the temporary directory
            _copier_config_file: the temporary copier config file

        Returns:
            the object instance, ready to copy !
        """
        # extract the template directory from the pytest command parameter
        template_dir = Path(request.config.option.template)

        # set up a test directory in the tmp folder
        (test_dir := tmp_path / "copie").mkdir()

        yield Copie(template_dir, test_dir, _copier_config_file)

        # don't delete the files at the end of the test if requested
        if not request.config.option.keep_copied_projects:
>           rmtree(test_dir)

..\..\..\miniforge3\envs\pytest-copie-dev\lib\site-packages\pytest_copie\plugin.py:144:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\..\miniforge3\envs\pytest-copie-dev\lib\shutil.py:750: in rmtree
    return _rmtree_unsafe(path, onerror)
..\..\..\miniforge3\envs\pytest-copie-dev\lib\shutil.py:615: in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
..\..\..\miniforge3\envs\pytest-copie-dev\lib\shutil.py:615: in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
..\..\..\miniforge3\envs\pytest-copie-dev\lib\shutil.py:615: in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
..\..\..\miniforge3\envs\pytest-copie-dev\lib\shutil.py:615: in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
..\..\..\miniforge3\envs\pytest-copie-dev\lib\shutil.py:620: in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = 'C:\\Users\\admin\\AppData\\Local\\Temp\\pytest-of-admin\\pytest-19\\test_template0\\copie\\copie000\\.git\\objects\\07'
onerror = <function rmtree.<locals>.onerror at 0x0000021E17614C10>

    def _rmtree_unsafe(path, onerror):
        try:
            with os.scandir(path) as scandir_it:
                entries = list(scandir_it)
        except OSError:
            onerror(os.scandir, path, sys.exc_info())
            entries = []
        for entry in entries:
            fullname = entry.path
            if _rmtree_isdir(entry):
                try:
                    if entry.is_symlink():
                        # This can only happen if someone replaces
                        # a directory with a symlink after the call to
                        # os.scandir or entry.is_dir above.
                        raise OSError("Cannot call rmtree on a symbolic link")
                except OSError:
                    onerror(os.path.islink, fullname, sys.exc_info())
                    continue
                _rmtree_unsafe(fullname, onerror)
            else:
                try:
>                   os.unlink(fullname)
E                   PermissionError: [WinError 5] Access is denied: 'C:\\Users\\admin\\AppData\\Local\\Temp\\pytest-of-admin\\pytest-19\\test_template0\\copie\\copie000\\.git\\objects\\07\\ca5c6677c34cf78b9dd7bd3a21dba891051fc5'

..\..\..\miniforge3\envs\pytest-copie-dev\lib\shutil.py:618: PermissionError
------------------------------------------------ Captured stdout call -------------------------------------------------

Initialized empty Git repository in C:/Users/admin/AppData/Local/Temp/pytest-of-admin/pytest-19/test_template0/copie/copie000/.git/

------------------------------------------------ Captured stderr call -------------------------------------------------
No git tags found in template; using HEAD as ref

Copying from template version 0.0.0.post3.dev0+ed582b2
 identical  .
    create  README.rst
 > Running task 1 of 1: ['C:\\Users\\admin\\miniforge3\\envs\\pytest-copie-dev\\python.exe', 'C:\\Users\\admin\\AppData\\Local\\Temp\\copier.vcs.clone.8zjlalsf\\_tasks.py']
Switched to a new branch 'main'
warning: in the working copy of 'README.rst', LF will be replaced by CRLF the next time Git touches it
=============================================== short test summary info ===============================================
ERROR tests/test_template.py::test_template - PermissionError: [WinError 5] Access is denied: 'C:\\Users\\admin\\AppData\\Local\\Temp\\pytest-of-admin\\pytest-19...
============================================= 1 passed, 1 error in 2.83s ==============================================

The coverage makes no sense

The tests are run in the special "pytester" plugin, it seems it's not registered by coverage and thus lead to 0% coverage.

understand why coverage is 0%

The current results of the pytest coverage results is 0%
It seems the repository is not detecting that the tests are calling the lib as it's all done under the hood in pytest itself.

If impossible to correct it will be better to drop this insight as it give a false idea of the quality level of the repository.

This package only work for templates with subdirectories

The main objective of the pytest-copie lib is to test advance template to make sure that they still work as expected when updating versions.

This requires the user to set-up the test suit outside of the template itself so the tested repository need to implement the _subdirectory in its answers.

This should be tested in the lib and also documented in the Usage page

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.