Code Monkey home page Code Monkey logo

framework-team's Introduction

Project boards for CMSSW framework development

Nomenclature

  • Task issue: A single work item, managed in the Tasks project.
  • Activity issue: High-level issue, could roughly correspond to an issue in cms-sw/cmssw. These are planned and managed in Activities project
    • Consists of many Task issues, or if it can't or doesn't make sense, it can itself be a Task issue as well.
  • Umbrella issue: Even higher-level issue to collect information on many related Activities. There is no specific planning for Umbrellas.
    • These are long-lasting development topics that consist of multiple Activities
    • Must have an ending, which makes Umbrella distinct from Overarching themes

The overarching themes are a rough classification of activities, and do not have a clear ending.

Maintenance Tasks are not part of any Activity, but have Task and Maintenance labels.

Documentation Tasks can be part of an Activity, or separate to any Activities. In either case, they have Task and Documentation labels.

Activity properties in the Activities project

Each activity should be assigned a priority. Each activity may also have a size, that is a very crude estimate` of the FTE time needed (will probably be incorrect, but that's life). The names of both priorities and sizes should be self-explanatory.

Status

  • New: Activity issue has been newly added to the project, priority has not yet been assigned
  • Backlog: Activity issue's priority has been assigned, but is not planned to be worked on in the current quarter
  • Ready: Activity issue is planned to be worked on, and in principle is open for anyone to work on, but the work has not started yet
  • Paused: Work was started, but got stopped for long time for any reason. Need to be followed up periodically.
  • In progress: Activity is being worked on
  • Done: Activity has been finished

Labels

Label colors have been created with https://colorbrewer2.org/

  • Priority: Breaking, High impact, Promised, Annoyance, Wishlist; 5-class YlOrRd (not to be used anymore, to be removed)
  • Kind: Bug, Optimization, Maintenance, New feature, Tool, User request, Documentation, R&D; 8-class Blues
  • Type: Umbrella, Activity, Task; 3-class Greens
  • Overarching theme: Throughput, Threading efficiency, Heterogeneous computing, Code modernization; 4-class Purples

Walkthroughs

Still to be done

  • What to do for a small maintenance fix?
  • How to update the plan of an Activity?
  • How to finalize an existing Task in an Activity?
  • How to add oneself to an existing Activity?
  • How to change status of an Activity or a Task?

Reference

Managing Activity issues

  • Create new issue (anyone)
    • On the right, select from "Labels" if it is a "Bug fix", "New feature", "Maintenance", or "User request"
    • From "Labels" select "Activity" (this should lead the issue to be automatically added to the Activity project with the "New" status)
  • Assign priority (manager, or in a meeting)
    • The priority is set within the Activities project. When the priority is set, the issue's status should be changed from "New" to "Backlog".
      • The priority issue labels should not be set anymore
    • There are many ways to set the priority and status, some examples
  • Planning on Activity issue (anyone)
    • The Activity issue is planned with a task list in the issue description
    • An Activity issue can be assigned to a developer at this stage, but it is not necessary
  • When an Activity issue becomes relevant to be worked on (ideally in the weekly meeting, can be in between as well; mostly for manager)
    • In the Activities project, move the status of the Activity issue from "Backlog" to "Ready"
    • Assign the Activity issue to one or more developers
    • Set the Quarter of the Activity issue to the current quarter
    • If the Activity issue has a task list, convert the elements of the task list that could be worked on to issues.
      • Click a circle at the right edge of the element. This creates a new issue
      • In the new issue, from "Labels" select "Task" (this should lead the issue to be automatically added to the Tasks project with the "Ready" status)
      • Set the Quarter of the Task issue to the current quarter
    • If the Activity issue consists from a single task, add the "Task" label to the Activity issue, which then becomes a Task issue as well
  • When the work on a Activity issue actually starts
    • In the Activities project, move the status of the Activity issue from "Ready" to "In progress"
    • Set the "Began" date to the date when the work started
    • Set the "Finished" date to the estimated end date
      • The estimate can be crude, e.g. end of current month, end of current quarter, end of current year etc.
  • When the last Task issue of the Activity issue has been closed
    • Close the Activity issue (the status is automatically set to "Done")
    • Set the "Finished" date to the date when the work was finished (e.g. last PR was merged, documentation was written, presentation was given)
  • If a Activity issue becomes dormant, e.g. no work in a few weeks
    • In the Activities project, move the status of the Activity issue from "In progress" to "Paused"
    • The paused Activity issues should be revisited regularly, e.g. to
      • Decide if/when the work could continue (keep the issue paused), or
      • If the remaining work is not that relevant/important anymore, in which case the Activity issue should be split
        • Move the remaining tasks into a new Activity issue with "Backlog" status and proper prioritization
        • Close the old Activity issue
        • Set the "Finished" date to the the date when the work was finished

Working on Task issues

  • When a developer starts working on a Task issue with "Ready" status, developer changes the status to "In progress"
    • If this is the first task of an Activity to be worked on, the status of the Activity issue should be updated as well (see above)
  • If for any reason the work on a task stalls for sufficiently long period of time (say more than week), set the Task issue status to "Paused"
  • Upon completion of a task
    • If the deliverable is a Pull Request
      • Change the Task issue status to "In review"
      • Link the PR to the task issue with one of the keywords in the PR description. While merging of the PR will not cause the Task issue to be automatically closed (because of the PR repository being in different organization than the project), having the Task issue linked to the PR is helpful.
      • When the PR is merged, either the developer or the manager closes the Task issue.
    • Otherwies, close the Task issue. The issue will be automatically moved to the "Done" column of the projeect board.
  • When the last Task issue of the Activity issue is closed, the Activity issue needs to be closed too (see above).

What to do when new quarter starts?

  • In the Activities project, update the quarter of all non-Done Activity issues that had the quarter set to the previous quarter
  • In the Tasks project, update the quarter of all non-Done Task issues that had the quarter set to the previous quarter

Earlier project instances

framework-team's People

Contributors

makortel avatar

Stargazers

W. David Dagenhart avatar Patrick Gartung avatar

Watchers

 avatar James Cloos avatar Dan Riley avatar Chris Jones avatar Malik Shahzad Muzaffar avatar W. David Dagenhart avatar Slava Krutelyov avatar Marco Musich avatar  avatar

framework-team's Issues

Figure out how to implement EventSetup consumes in OscarMTProducer

See cms-sw/cmssw#32011, done in cms-sw/cmssw#34031

Decided to go an easy way. The handling of the SensitiveDetectors was the most difficult as which plugins to load (and therefore which EventSetup data products to use) was being determined at begin Run time based on reading another EventSetup data product.

Solution now is to break SensitiveDetectors into two parts, the SensitiveDetector and a builder where only the builder reads from the EventSetup. All builders are loaded at OscarMTProducer construction time and are passed the ConsumesCollector.

The OscarMTProducer's infrastructure has been change to accomodate this and one SensitiveDetector was converted as a demonstrator. Further changes should be handled by the simulation group.

Store (vector) architecture in "event provenance"

See cms-sw/cmssw#30044. The "event provenance" should be interpreted widely along "metadata available at Event transition functions".

Possibly depends on ProcessBlock

Progress being tracked in https://github.com/makortel/framework/milestone/22 (opened milestone as the development started to require many tasks)

Needs first

TODO list for next steps

  • Check how the existence of Transformer looks in the event-level provenance
  • How does ProductProvenance map to data products? Is it per data product or per module?
    • Seems like it is saved per BranchID. So we could make Transformer to add information there.
  • One alternative way would be to add "not identical but equivalent" concept for PSets. Keep the "identity" PSet in the PSetRegistry for Runs and Lumis, and add "equivalence" PSet for per-event information (replace or add additional ProcessHistoryID to EventAuxiliary). Would have to add a concept between "tracked" and "untracked" parameters that would be stored only in the "equivalence PSet" (but do not expose this functionality publicly)

tbb::task_group: Make initial pull request to test full CMSSW

  • Make initial pull request to test full CMSSW
    • Fix compilation problems found by pull request
    • Fix runtime failures in pull request
      • Wait for preview features to be turned on in tbb build
      • Rebase onto master to use new tbb preview features
      • Implement use of preview features
      • debug end of job seg faults; split to #59
      • debug seg faults using FWLite EventSetup unique code

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.