@ryankeairns commented on Wed Oct 11 2017

Relates to #3678

From the conversation in issue #3678 , we're splitting out the concept of bubbling up errors to this new issue.

As a user, I want to quickly know why my build failed
so that I can correct the issue
and get to a successful build outcome.

If your build fails (regardless of the trigger), let's pipe the failure reason back into the UI on the build output page. So, not only display it in the streaming build output, but above the output box as a separate HTML element/banner/message.

Specific examples would be things like:

• Incorrect Dockhub username or password
• Origin keys not found
• Origin in plan file does not match this origin

To-dos

• UX: create mockup of build job error message in UI
• Dev: Report errors to the frontend end via API
• Dev: Add message to UI per mockup

@apriofrost commented on Wed Oct 11 2017

This idea is so brilliant!
If we have docs that we can link to in the UI to explain possible ways to resolve the error, that will be great!

@tashimi commented on Thu Apr 27 2017

When a user has a package that has dependencies that have more recent builds than the user's most recent build of their package, the UI should show a visual cue to indicate that the user needs to take an action to rebuild their package and take advantage of this newer version.

This will require API & front end work.

@ryankeairns commented on Thu Sep 21 2017

@tashimi can we remove the Pre-Neighborhoods milestone from this issue (UI alerts)?

@tashimi commented on Thu Sep 21 2017

@ryankeairns Sure, but let's create a follow up milestone because notifications is important and should be done short term (I would prefer to have it done by the end of October)

@ryankeairns commented on Thu Sep 21 2017

Done, added an On Deck milestone.

@tashimi commented on Mon Oct 02 2017

Fletcher would like to request a "Builder kicked off a build of xxx" in Slack

Aha! Link: https://chef.aha.io/features/APPDL-14

@lilianmoraru commented on Mon Jul 11 2016

Currently there is no way to remove a package(unless there is good reason).

I propose habitat to support unlisting packages, like crates.io does.
The idea is that some packages might be broken or some other issues might appear with them.
The person that published that package might release another version with a fix and desire to unlist the broken package, currently this is not possible.
After unlisting, the package will still be present and work for those that specifically select to use that version, but it will not be available/visible to new users.

@eeyun commented on Tue Dec 20 2016

This is absolutely something that we're going to want to take on. Thank you for opening the issue and we'll come back to close this circle when we have it implemented.

@lilianmoraru commented on Tue Dec 20 2016

I gave these examples but the main reason I opened this was because there were to many old versions of packages and it was really confusing when trying to search for a package(example: search for gdb).

@gfodor commented on Sat Sep 23 2017

+1 for this, we had a mis-naming of one of our packages ("libopus" vs "opus", a redistribution of the opus codec) and now we have the "libopus" (wrong) package floating around. not the end of the world but confusing until we can unlist it.

Aha! Link: https://chef.aha.io/features/APPDL-76

@mwrock commented on Sun Oct 08 2017

When a user lands on a "connect a plan" view and has not installed the builder app on their org, they will not have their repos recognized...we know that. We have an instructional box informing the user to install the app if they have not done so. It provides a link to the app that opens in a new tab. So if the user proceeds to install the app ina new tab, the connect a plan tab will still not see their repos until they reload the view. We should add a sentance to the efect of: "Reload this page after installing the app." And have "reload" be a link to actually reload.

There is likely a better experience to be had here but this may be the quickest fix for now.

@cnunciato commented on Fri Oct 13 2017

@ryankeairns thoughts? Still a couple of rough edges here; we could probably poll GitHub every few seconds looking for an install, and kick in automatically when either an installation or the the repo you've selected turns up.

@tashimi commented on Thu Apr 27 2017

For https://chef.invisionapp.com/share/EU9BOXKF6#/screens/229781862, some of this data cannot currently be accessed via the builder API. We need to add those API endpoints and update the UI to render this data dynamically as it changes.

@ryankeairns commented on Thu Sep 21 2017

@tashimi Same here, are these updates to the Explore page still in scope for Pre-Neighborhoods?

@tashimi commented on Thu Sep 21 2017

@ryankeairns I think this is a nice to have, we can manually update with interesting content for the meantime

@chefsalim commented on Sun Oct 15 2017

Currently, an unsharded service that sets it's 'shards' config to an empty list is unable to properly route messages. Need a proper solution so we don't have to iterate through 128 shards when only a single shard exists.

@raskchanky commented on Wed Aug 30 2017

Is there a way to create a new API endpoint that mashes together the information present in:

• /pkgs/core/redis/3.2.4
• /projects/core/redis/jobs

Ideally you just hit 1 endpoint, that can be paginated in the future, and you get all the information you need to know about builds that have succeeded and failed, complete with version information.

You'd want the information present in /projects/core/redis/jobs as a starting point. Then for each item, you'd also want to know:

• What channels is this release in?
• Who initiated this build?
• What platform is this release for?

@reset commented on Wed Oct 11 2017

Now that we've got the Docker exporter tied into Builder and publishing to the Docker Hub working it's time to add the other major container registries. This should largely be UI focused with the remainder of the work being in validating credentials which may need to be done server side depending upon each provider's CORS support.

• Amazon Container Registry - #3743
• Google Container Registry - #3745
• Azure Container Registry - #3744

To-Do's

• UX: Mockups/Prototype for Integrations tab w/multiple registry accounts
• UX: Mockups/Prototype for Connect a plan file enable publishing w/multiple registries
• Define form fields (repo and tag format) for each account type (Docker, GCP, Azure, Amazon)
• UI: Add new buttons to Integrations tab
• UI: Update help text below label/heading and on delete confirm dialog
• UI: Display list of accounts on 'Connect a plan file' step 2 when publishing is turned 'ON'
• UI: Clicking 'configure this account' opens dialog with form (publish settings)
• UI: After configured, the 'stop sync' icon would disabled publishing for that account (i.e. remove settings; back to original state of 'configure this account')
• UI: After configured, the 'gear' icon would re-open the dialog with form pre-filled so user can edit settings.
• UI: Switching publishing to OFF hides the list of accounts and disabled all publishing
• Credentials verification

UI Mockups

1. Prototype for Integrations tab w/multiple registry accounts
2. Prototype - publish to multiple registries

@ryankeairns commented on Mon Oct 16 2017

@reset would users need the ability to export and publish to multiple of these per package?
In other words, when I connect a plan file to builder, could I enable Docker and Amazon, for example?

cc: @elliott-davis

@elliott-davis commented on Mon Oct 16 2017

Also, do I have the ability to support multiple regions of each of these registries?

cc: @ryankeairns

@tashimi commented on Mon Oct 16 2017

Yes to both

On Oct 16, 2017, at 9:48 PM, Elliott Davis [email protected] wrote:

Also, do I have the ability to support multiple regions of each of these registries?

cc: @ryankeairns

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@fnichol commented on Mon Oct 16 2017

Beat me to it: yes to both 😄

We designed the "integrations" to be a list so that there could be multiple types (docker hub, ami), and multiple instances of the same type (ami/us-west-2, ami/us-east-1, etc)

@ryankeairns commented on Mon Oct 16 2017

@ryankeairns commented on Tue Oct 17 2017

Prototype link added to issue description for the Integrations tab. Next, working on prototype for the multiple registry targets in the 'connect a plan file' flow.

@ryankeairns commented on Tue Oct 17 2017

What do you all think of this approach for publishing to multiple registries when connecting a plan file? I've re-used design patterns that exist elsewhere in the app (e.g. the table list and modal dialogs) with thoughts towards varying config settings based upon registry type (e.g. tag fields would change, etc).

When you 'turn on' publishing, you would see a list of all the registry accounts that you've configured on the Origins > Integrations tab. From this list, you could activate/configure as many as desired.

In this mini prototype, you can activate Docker Hub publishing, then remove it.
Prototype - publish to multiple registries

@reset commented on Wed Oct 18 2017

@ryankeairns I think it looks great - I think it's just fine for now to publish to only one of each type but I think in the future people may want to be able to publish to n number of Docker Hub repositories

@ryankeairns commented on Mon Oct 16 2017

As a Habitat team member,
I would like to capture data that helps me determine where users tend to get stuck
so that I can improve the experience of getting started with Habitat.

With analytics tools, we can track key events to find both challenging flows that make it difficult for users to see immediate value from Habitat. In order to target specific events, it would be helpful if buttons were uniquely identifiable through and id parameter and/or unique class name.

Currently, several buttons do not use either which makes them difficult to discern from one another in our data analysis.

• Add a unique id or class parameter to all buttons in the Builder UI

List of buttons:

• Sign in with Github
• Connect a plan
• Next (to step 2 of connect a plan file)
• Save connection
• Create an origin
• Generate key pair
• Add Docker Hub account
• Build latest version
• Send invitation

@reset commented on Tue Aug 08 2017

We need a mechanism for notifying external systems of build and publish status for packages. This is often manifested in an HTTP callback known as a WebHook.

We should aim at a minimum to support WebHook notifications for

• Build finish - with a payload containing the status and information about the generated artifact
• Publish finish (i.e. Docker Hub export) - with the same payload as above

@tduffield commented on Wed Oct 11 2017

I would also be interested in a promotion notification.

@ryankeairns commented on Mon Oct 16 2017

For automated rebuilds of packages connected to Habitat Builder:
As a user, I want to know when my package has been rebuilt by Builder
so that I can test and deploy the updated package.

For non-connected builds where your deps have changed and you should rebuild rebuild:
See #2185

Currently, I would have to manually/periodically check the UI to determine if there are new versions of packages (or deps of my package) available. It would be nice if, as soon as my package is rebuilt, I receive/see a notification (email also?) letting me know that it was rebuilt and why it was rebuilt (e.g. "Fix for CVE").

If my package (plan file) is not connected to Habitat Builder, then I receive/see a notification when one of my package deps has been updated; instructing me to rebuild my package.

@ryankeairns commented on Mon Oct 16 2017

@cnunciato feel free to add more details; just added enough so that we don't lose track.

@rsertelon commented on Tue Oct 17 2017

Maybe a reason for builds could be nice too (in the builds tab for a package)?

For example knowing if build was started by a member of the organization, or following a commit, or even automagically by builder services because of dependency update.

@reset commented on Tue Oct 17 2017

@ryankeairns yesterday @christophermaier, @fnichol and I had a brainstorming session about some things we'd find useful in this feature. Specifically we think it would be really useful if there was a "commit message" when promoting builds from unstable to stable and we think that the action of promotion is what should kick off builds in other origins. If we were to do this, you could submit a message like "Fix for CVE " or include the entire changelog of a software's release in your promotion message. This could be a really neat way to surface release notes :)

@raskchanky commented on Thu Sep 07 2017

The Origin protobuf has gotten a little bloated and unwieldy. The Origin itself should just contain id, name and owner_id. The other fields (private_key_name and default_package_visibility as of this writing) should be separated out into OriginSettings, or something similar and future additions to Origin should happen there.

Aha! Link: https://chef.aha.io/features/APPDL-75

@jtimberman commented on Fri Apr 07 2017

Scenario: I have a private Habitat depot where I want to publish my application, acme/myapp. This application is written in Ruby on Rails, and I want to use the core packages as dependencies without rebuilding them or mirroring the public depot to my internal one. I'm okay with my application nodes talking to the public depot, so I would like to use it as a fallback for any packages that can't be found locally.

I propose adding a command-line option to hab pkg install that will use a fallback URL as the depot for missing packages. This option would have a corresponding ENV variable so it wouldn't need to be specified. So for example, if I do:

hab pkg install acme/myapp --url https://private-depot.example.com/v1/depot --fallback-url https://willem.habitat.sh/v1/depot

It would download acme/myapp from my private depot, private-depot.example.com/v1/depot, and any dependencies such as core/ruby or core/glibc from willem.habitat.sh/v1/depot.

I think the "sane default" is for this fallback URL to use the public depot. Perhaps a CLI option and ENV variable to disable this for air-gapped environments that can't talk to the internet.

@jtimberman commented on Fri Apr 07 2017

Longer term we will want to implement "pull through proxy" and federation to replace this, but this is fine for an initial spike.

@eeyun commented on Tue Apr 11 2017

There will certainly come a point where we're going to need to answer this use-case. For now we're going to flag this as help wanted as we have some other work that would supersede being able to start on this.

@tashimi commented on Thu Sep 28 2017

We need to add the command hab pkg export cf <pkg name> to the UI under "export command", for example: https://bldr.habitat.sh/#/pkgs/core/erlang16/latest

@cnunciato commented on Fri Oct 27 2017

@tashimi Should this be limited to packages that are also services (or some other criteria), or should it be visible to all packages?

@tashimi commented on Fri Oct 27 2017

@cnunciato that is a great point, we should totally limit it in that way

@cnunciato commented on Fri Oct 27 2017

Ok, we’ll restrict it to services only then, as we do with the others.

@chefsalim commented on Thu Jul 06 2017

1. We need automated disk and database backups
2. We need a reliable way to do snapshots (if snapshots are the backup strategy)
3. Look into implementing Postgres DR / backup solutions (someone mentioned S&W's shield last week)
4. Investigate running Postgres in HA mode - don't rely on a single instance (another S&W follow-up)

@adamleff commented on Fri Sep 15 2017

A while back, we decided to move InSpec from core/inspec to chef/inspec since it didn't make much sense to have it live as a core package, and none of the InSpec maintainers who control the release are core maintainers. However, since core/inspec still exists (because we all agree removing packages from the depot is A Bad Thing(tm)), it recently caused a problem when trying to automatically rebuild the world for the recent Ruby CVEs.

I'd like to propose a mechanism for marking a package as deprecated such that bldr doesn't try and rebuild it. Bonus points for the ability to de-list the package (while still having it available as an artifact), and double-bonus points for the ability to redirect a user from package A to package B so they can find the new hawtness.

Sorry if this is a duplicate. I know this has been chatted about before and I tried to find an existing issue unsuccessfully. Thank you!

@tashimi commented on Sat Sep 16 2017

Agree! This is something we need to add.

@lilianmoraru commented on Thu Dec 28 2017

Related issue #1051

@chefsalim commented on Fri Aug 04 2017

We can get into a state where the syslog can fill up the system disk on our builder instances.
This card is to set up a syslog limit when provisioning instances via our Terraform scripts.

Set the following in /etc/systemd/journald.conf :
SystemMaxUse=500M

@tashimi commented on Mon Oct 02 2017

It would be great if we could add a "short" and "long" description to Habitat packages published to Dockerhub including a short description of the package, that it is built with harts (?) by Habitat Builder, and a link back to the correct Habitat Builder page. This will increase visibility and discoverability of the Habitat Builder service and be a better user experience.

@fnichol commented on Wed Oct 04 2017

This will take some research, but I'm hopeful!

@chefsalim commented on Thu Jun 01 2017

Currently, the scheduler service has internal logging that can be used to generate gantt charts of job scheduling progress. This would be a good feature to build into the UX for builder admins and other stakeholders.

@raskchanky commented on Tue Sep 26 2017

This is related to habitat-sh/habitat#3271

We can save some time traversing shards during account/session creation and retrieval if we create a fat session token that holds all the data we care about and store that, instead of storing the GH token directly.

@chefsalim commented on Thu Jul 06 2017

Currently we are storing the package repository on disk. We should get them into S3 in order to reduce the incident surface area and reduce backup/restore tasks.

@mwrock commented on Tue Aug 29 2017

also see #1933 for more context

@reset commented on Tue Aug 15 2017

As an owner of an origin, I need a way to transfer ownership to another member, so I can leave the origin or entrust it to somebody else

@ryankeairns commented on Thu Sep 21 2017

Mockup for transferring ownerhisp

Note that there must always be one owner.
Note that we discussed setting the origin creator of the origin as the owner (we'll need to go back and retroactively do that for current origins).

^ There is overlap with #2943 - not sure which issue will cover that work, just calling it out for awareness.

Aha! Link: https://chef.aha.io/features/APPDL-58

@tashimi commented on Thu Apr 27 2017

When a user finds a package or dependency that they want to bookmark, they should be able to click a "hart" icon that creates a list of favorites they can browse and edit later.

@ShalokShalom commented on Tue Jun 06 2017

You mean a heart icon?

@rsertelon commented on Sat Nov 25 2017

Could be the hab heart we see on slack ;)

@chefsalim commented on Sat May 13 2017

Currently, in order for the builder service to auto build a plan, it needs an origin Project that contains metadata such as the repository and other plan location info. The origin project needs to be explicitly created before a package can be successfully built by builder.

This issue is a proposal to automatically create the origin project during package upload if the project doesn't already exist. The package plan would need to contain sufficient metadata for this. We could also elide the metadata for core packages.

@christophermaier commented on Wed Sep 06 2017

Most services that make up Builder run more than one thread of execution, but logging output does not reflect this, making it difficult to determine which thread a given message is coming from.

@julian7 commented on Wed Jul 06 2016

Currently hab-depot is tied to GitHub oauth. It would be good to allow alternative auth sources for private depots (like github enterprise, gitlab).

@adamhjk commented on Wed Jul 06 2016

This seems like a great idea, especially as we think about people deploying depot's locally.

@tdavis commented on Wed Jul 06 2016

I was about to request this, actually. As a related but possibly easier first step, it'd be nice if you were to complete the api.raml specification to make implementing the depot easier. The RAML file doesn't appear to formalize the authentication scheme, either.

@tdavis commented on Wed Jul 06 2016

After looking at the code, it seems a bit more complicated than that since the HTTP API is wrapping calls to a bespoke RPC service elsewhere which also depends on some protobufs defined for builders?

@reset commented on Thu Jul 07 2016

The depot is mounted in the builder-api which is a gateway server. A gateway server does external authentication at the edge to then find and create you an account on the session server. Any of the components prefixed with builder-* are part of a hosted service that will be launching in the near future. Services communicate to each other over a binary protocol using ZeroMQ and Protobufs. Messages are sent from one service through a routing server and into another service.

Right now the accounts are tightly coupled with GitHub so we can get a first version of the service out the door. There will be an abstraction to allow people to develop different authentication mechanisms at the edge but that'll happen after the first version drops.

Be prepared for an adventure if you want to add pluggable back-ends before then 😉. It's do-able but you'll need to understand the whole bowl at this point!

@tdavis commented on Thu Jul 07 2016

Any of the components prefixed with builder-* are part of a hosted
service that will be launching in the near future.

Okay, that clears up a lot of my confusion around the purpose of all
those components.

Services communicate to each other over a binary protocol using ZeroMQ
and Protobufs. Messages are sent from one service through a routing
server and into another service.

Out of curiosity, why did you choose to implement your own protocol over
ZeroMQ over using an established RPC layer like Thrift or gRPC?

Be prepared for an adventure if you want to add pluggable back-ends
before then 😉. It's do-able but you'll need to understand the
whole bowl at this point!

That's pretty much what I figured, thanks for clarifying things. I'm in
no great hurry so I'll wait for things to flesh out a bit more.

Habitat is the first automation tool I've seen in a long time that makes
any damn sense so I'm really excited for it and will gladly contribute
when things have stabilized.

@reset commented on Thu Jul 07 2016

@tdavis ZeroMQ and Protobufs are both established and well documented open source libraries that accomplish nearly the same thing. I could have chosen a number of other combinations but if I had more time I probably would have written my own sockets library and not used ZeroMQ, either. I'm happy with ZeroMQ now that I've had plenty of time with it, but I have preferences about how the API should look and feel and a lot of the popular open source libraries don't match them.

The documentation around the builder services will get better as they get closer to completion. It should be pretty easy for developers to contribute as the services all behave as single threaded servers even though they are highly concurrent.

I'm glad you're excited about Habitat and that it makes sense to you! It's only going to get better as it matures :)

@adamhjk commented on Thu Jul 07 2016

@tdavis to riff on what @reset said - there are specific behaviors about the way the services connect to one another that are facilitated by ZeroMQ at the socket layer. Using something like Thrift or gRPC wouldn't have given us those (although we could have written our own socket layer, as @reset points out.) Protobufs work pretty great for what we use them for - back-compat binary serialization. :)

@chefsalim commented on Tue Sep 26 2017

Currently, Builder uses the package run-time dependencies (DEPS/TDEPS) to build it's reverse dependency graph. The dependency data comes from the origin_packages table.

The fact that we only trigger on runtime deps means that there are cases (eg, core/hab) where we won't automatically build on a large build (like openssl) because it doesn't have any runtime deps.

This issue is about discussion of expansion of the dependency graph by including both build dependencies, and dependencies for composite plans (which will likely have a different format).

Things to take into account:

• Data model changes and ingestion into builder
• Potential introduction of circular dependencies
• Impact of expansion of build graph
• Scalability / build resources required to support the expanded graph
• Other unknowns

Aha! Link: https://chef.aha.io/features/APPDL-59

@fnichol commented on Tue Oct 03 2017

Currently the Docker Engine's data directories aren't cleaned up when the job is torn down. It shouldn't be big at the moment as the exporter cleans up its image layers.

@fnichol commented on Wed Oct 04 2017

This is non-critical and only applies to hosted Builder. We'll fix this shortly, but doesn't impact exporting Docker images.

@reset commented on Mon Oct 09 2017

We have the ability to remove members from an origin but we don't have the ability to leave one ourselves. We should add a button on the show origin UI component which allows you to leave an origin if you aren't the owner

@raskchanky commented on Mon Aug 07 2017

There are places in the codebase where we're making assumptions about our database shards that aren't going to work out once we have more than one database server. Searching for packages is one example. Right now, the expectation is that when you search the depot for "redis", you see every redis package for every origin. We shard on origin, though, so in order to retrieve results for all origins, we build up a query and run that query on every shard. This works right now because the entire database is on one machine, but once that's not the case, this will fail.

I'm guessing there are probably other places as well, and it would be good to find these problems sooner rather than later. One way to do this would be to setup our acceptance environment with a minimum of 2 database servers.

@ShalokShalom commented on Tue Oct 10 2017

People, who want to see the package list do not see the wonderful Explore page.
Can we do some Google magic, to increase this chance?

@eeyun commented on Tue Oct 17 2017

Yes. This is an excellent find. We need to spend some time on our SEO to be certain! Thank you for opening this issue!

@chefsalim commented on Mon Oct 09 2017

Currently, when we build in Builder we set the HAB_BLDR_CHANNEL to prioritize picking packages from that channel. If a package is not found there, we pick it up from stable instead. This is normal, however the logging makes it appear that something bad happened. Would be good to have more intelligent logging.

kibana: Resolved build dependency 'core/coreutils' to /hab/pkgs/core/coreutils/8.25/20170513213226
» Installing core/gcc from channel 'bldr-824744994724225024'
∅ The package does not have any versions in the specified channel.
∅ Did you intend to install one of the folowing instead?
∅ core/gcc/5.2.0/20170513202244 in channel stable
∅ core/gcc/5.2.0/20170513202244 in channel unstable
✗✗✗
✗✗✗ Package not found
✗✗✗

Aha! Link: https://chef.aha.io/features/APPDL-60

@reset commented on Wed Oct 11 2017

Add post-processor publishing support for Azure's Container Registry to Builder

#3742

@jtimberman commented on Tue Aug 09 2016

As a Habitat user with network restrictions or other requirements, I want to run a Depot on my local network. To do this, I need to download all the core dependencies from the public depot or build them myself. The former is preferred as it will require less of my time, care, and feeding, and ensures that I can use known good builds of all packages.

UX ideas

# mirror the entire public depot, including all origin public keys
hab depot mirror

# download a single package and its dependencies
hab depot download core/pkgname

# download an origin public key
hab depot download core

Open to other ideas or feedback. This would go nicely with our internal work to make the Depot easier to run for people.

@bdangit commented on Tue Aug 09 2016

yes! 👍

@reset commented on Tue Oct 10 2017

If a user renames their GitHub account and then attempts to login they are unable to. We need to store a mapping of external IDs to internal account IDs on the SessionSrv which holds the session (keyed by external ID). This will allow us to route by account ID instead of by account name if a mapping is found ignoring what GitHub believes their account name is.

Aha! Link: https://chef.aha.io/features/APPDL-61

@robbkidd commented on Wed Oct 11 2017

I was invited to an origin a while back (Spring? May-ish?). I'm pretty sure I accepted it at the time.

Today, I see that I am a member of the origin and can definitely do things an origin Member can (I added a key, I connected a GitHub repo to a packge). But I've got a lingering invite that I don't know what to do with:

Last week, I only saw the invite which would error if I attempted to accept. I could not interact with the origin.

@scotthain commented on Fri Oct 13 2017

Doesn't seem to have any ill effects for me but I'm also seeing this, with a similar situation to Robb here. (I can confirm that I did accept the inv to the origin back in Apr/May-ish)

@rsertelon commented on Sun Dec 10 2017

@robbkidd @scotthain Do you still see the lingering invite? I've dug up in the code a little, but cannot find reasons for it to be shown.

@robbkidd commented on Sun Dec 10 2017

Yep. My origins list still appears as shown in the screenshot in this issue.

@scotthain commented on Mon Dec 11 2017

Likewise! Very strange :D

@rsertelon commented on Mon Dec 11 2017

Can one of you invite me (temporarily of course) to one of your origins? So we can test that it will still happen or if your lingering invites are just quirks that would need a one-time database update :)

@stevendanna has been invited and his invitation disappeared as expected.

I suspect there were a bug in the invitations process that might have been fixed since you accepted your invitations. A "simple" SQL query might be able to fix this?

@robbkidd commented on Mon Dec 11 2017

Yea, my money is on @scotthain and I having invitations in the database from before the roll-out of builder. It's probably something builder admins can clean up in the data as opposed to a bug in the current code base.

@scotthain commented on Mon Dec 11 2017

I would 100% agree with this - we were very very early adopters (before there was a UX for accepting, IIRC) and this is likely just cruft if you can't replicate it.

@tashimi commented on Fri Apr 28 2017

When a user looks at an origin/package, if that origin/package has a ReadMe, that ReadMe should be displayed to provide information such as:

• what is the package
• who authored it
• its contents
• how to use it (do you call it as a dependency of your application, or do you need to add additional plan & configuration to use it)
• its expected use
• scaling up/scaling down behavior
• clustering behavior (if available)
• what habitat topology it should use
• what is the best update strategy for different deployments
• how to monitor the health of the service at the application level
• is an HA configuration possible? if so how

To-do

• Update the generated README with instructions for the additional items above (see #2571 )
• Add the README as a tab in the UI package view (this issue)

@ryankeairns commented on Thu Jun 08 2017

We should also provide a link from the package view to the plan on github (regardless of whether it has a README).

@ryankeairns commented on Mon Jun 12 2017

Related, we would also like to generate a templated README via hab plan init - see issue is #2571

@ryankeairns commented on Mon Jun 19 2017

Prototype for a revised package detail view that includes README content.
https://chef.invisionapp.com/share/3ZBKYLQTW#/239684122_package_Detail_V2

@ryankeairns commented on Mon Jul 24 2017

@tashimi FYI, here's the issue for adding READMEs in the web ui (that we discussed wrt UX priorities). There's a link to the prototype included here as well and it's currently in the Backlog.

@ShalokShalom commented on Mon Jul 24 2017

THIS

@rsertelon commented on Sat Nov 25 2017

#4077 shoud fix templated README generation :)

@ryankeairns commented on Tue Dec 12 2017

Updated the description...
There are two parts to this effort - 1) add additional topics to the generated README #2571 and 2) display the README in the web app UI (this issue).

@chefsalim commented on Thu Jul 06 2017

We should be able to specify a branch or a tag for builds that are kicked off via the Builder.
Currently, the site shows only a 'Request Build' button without being able to customize anything.

@chefsalim commented on Wed Jan 24 2018

@ryankeairns commented on Tue Jul 25 2017

When viewing build output, the channel tags should appear in the build metadata section above the streaming output.

A mockup of the desired UI design is here:
https://chef.invisionapp.com/share/3ZBKYLQTW#/240518817_channels_Mvp_-_Build_Output

@burtlo commented on Tue Oct 03 2017

As a Habitat developer 
Given that I am building ImageMagick
When I configure and build ImageMagick 
And it reports that the file `jerror.h` cannot be found
Then I want to run a command like `hab pkg provides jerror.h` 
And see a list of packages in the depot that contain the file `jerror.h`.

I am new to building software from source. So when I see a configuration script not find a particular file or build macro (autoconf) I don't know immediately where to look for it. I know that the core packages are using some naming conventions similar to what I would find in many Linux distros so searching the Internet sometimes gives clues. But there are enough variations that it makes it hard to know.

In this situation I describe above it is pretty clear that I need a JPEG library because ImageMagick makes it clear:

-------------------------------------------------------------
checking for JPEG...
checking jconfig.h usability... no
checking jconfig.h presence... no
checking for jconfig.h... no
checking jerror.h usability... no
checking jerror.h presence... no
checking for jerror.h... no
checking jmorecfg.h usability... no
checking jmorecfg.h presence... no
checking for jmorecfg.h... no
checking jpeglib.h usability... no
checking jpeglib.h presence... no
checking for jpeglib.h... no
checking for jpeg_read_header in -ljpeg... no
checking for JPEG library is version 6b or later... no
checking if JPEG package is complete... no
-------------------------------------------------------------

For other packages that I have attempted to build I have often times not known where to start.

I solved the above situation by using hab pkg search jpeg, installed the two core jpeg packages (libjpeg-turbo and openjpeg). Then used hab pkg provides jerror.h and was able to find the file I needed. I am suggesting this strategy to others in the documentation I am creating. However, this strategy will not always work or may require you to boil the ocean hab pkg install EVERYTHING to perform a hab pkg provides.

@reset commented on Tue Oct 03 2017

@burtlo this is an enhancement that we'll definitely implement

@chefsalim commented on Thu Jul 06 2017

1. We need to have an operations run-book that walks through typical operations on the production site - eg, re-creating / re-depolying builder nodes, recovering from datastore or package backups, etc.
2. The run-book should include how to communicate outages
3. It should also include information on alternate communication mechanisms for the team (eg, in case Zoom is not working)
4. It should include procedures for AWS and Terraform tasks as well
5. Run-book tasks should be regularly practiced / practicable.

@nellshamrell commented on Mon Jul 31 2017

Update -

I've got a new, basic builder environment terraform config working in a branch of the cloud_environments repo. I still need to add in instructions for setting up github, testing builds, etc.

@bixu commented on Wed Oct 11 2017

Right now hab pkg search returns only 50 entries total, which makes getting/parsing my complete package list programmatically (via cli) impossible.

@rsertelon commented on Sat Nov 25 2017

The API seems to limit results to 50 at most by design.

To get this feature we'd first need to allow more than 50 results at once.

@bixu commented on Sat Nov 25 2017

Correct. Maybe there’s a performance reason for this, but I expect most CLI workflows to happily handle several thousand lines of output from a call.

Plus, if we do need a limit, showing the newest first would be nice :)

On 25. Nov 2017, at 22:12, Romain Sertelon [email protected] wrote:

The API seems to limit results to 50 at most by design.

To get this feature we'd first need to allow more than 50 results at once.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@tashimi commented on Mon Oct 02 2017

When Builder performs an automatic publishing step, we should add a "run" command to inform users how to run that published package from where it lives. i.e. docker run <dockerhubid>/<packagename> which would run the latest package, or run an exact image by adding a tag.

cc @ryankeairns

@rsertelon commented on Sun Nov 26 2017

Linked to #3385

@cnunciato commented on Tue Dec 19 2017

One thing to keep in mind is that while we know that an integration exists for a given package, we don't (as of today anyway) track whether an image has been published with that integration, so that command stands a good chance of failing. Also, since it's the project settings that contain the external repository details, and that API is (at least currently) restricted to origin members, only origin members would be able to use this feature, unless we were to build some additional APIs for public origins & packages.

@reset commented on Wed Oct 11 2017

Add post-processor publishing support for Google's Container Registry to Builder.

#3742

@cnunciato commented on Fri Oct 06 2017

We need a way to modify the settings of a project/plan-connection that doesn't yet have any packages. This can be handled in the Settings tab -- we just need the UI to provide a way to navigate to that.

@cnunciato commented on Fri Oct 06 2017

cc @ryankeairns

@ryankeairns commented on Fri Oct 06 2017

Adding link to mockups for this... in it you can:

1. view a pending packages (ie. plan connected, but not available releases yet)
2. view a package that has only manual uploads, just for comparison purposes

To-do

• Add a new table above Packages that lists projects which have not .harts in the depot yet.

Mockups

Pending packages

cc:/ @raskchanky

@ryankeairns commented on Sat Oct 07 2017

This is good for launch, but I’d like to keep it open until it more closely resembles the mockup. We can move it to On Deck.

Nice work knocking this out last minute @raskchanky !

@raskchanky commented on Fri Sep 29 2017

Right now, the builder-api rdeps_show handler does not respect private packages. If you call that with a package identifier, it will fetch all of the reverse dependencies, even if those include private packages that you are not authorized to see. The only place this API call is used is in the hab bldr job start command, where it is used to print out a list of all the jobs that will be scheduled if you start a group.

I attempted to solve this problem already, and it is a delicate and sticky situation. We should talk through how this should work.

@eeyun commented on Tue Jun 27 2017

There's an issue in a subset of non-semantic version sorting at build-time.

The only packages I've seen exhibit this are specifically the oracle versions within the JDK package.
The most recent example popped up with core/jdk8/8u131.

Repro: If you create a package with pkg_deps=(core/jdk8 core/tomcat8) the collision will occur at build time in which the "latest" version of core/jdk8 that gets returned is in fact NOT the latest version and as such will conflict with tomcat. The jdk8 package will instead return core/jdk8/8u92.

I haven't had a chance to look through the code to try to deduce what the actual root cause is. But I think just from looking through the UI it looks like its possibly related explicitly to the way we handle sorting of available packages/builds. If you hit the depot and look at the available jdk8 packages they get returned in the order below:

core / jdk8
package

VERSIONBUILDSLATEST
8u92 3 2016-06-20
8u131 1 2017-06-22
8u111 4 2017-05-14
8u102 2 2016-10-31

@rsertelon commented on Sun Nov 26 2017

I've dug into source and found the function that sorts versions.

The documentation of the method clearly explains why java versions sorting fails. They are sorted lexicographically (as one would suspect).

Maybe we could support other separators for version numbers, but it's hard to define which ones. To make it work for java, u must be one of them. . is obviously one too. Maybe all unique characters that are not numbers could be used as separator?

Don't know if there are other versions than java that have this problem though. Could someone in core team check if there are version numbers without . in bldr data? I feel like this might be the only case...

If so, they changed the version scheme from java 9 onwards, so is it worth modifying our version sorting policy for this case? Maybe we could instead change java versions in core plans (although it may not feel right either ^^).

To find the list of origin keys I have, I need to currently do:

hab origin key export msuriar --type=public

This is annoying. It would be nice to be able to just list all the origins I have keys for, and the - values for them.

@apriofrost commented on Thu Jun 22 2017

People make typos and mistakes, our UI should support 'undo' when it doesn't cause other usage issues -- such as if there's no packages and no neighborhood connected.

This may be particularly important when a new Habitat user go through the new 'download and install' steps, which will guide them to the builder UI to create an account and the first origin.
The reason why we want to guide them to the UI (instead of the CLI) to create the origin is that we can validate if the origin is not taken by others upfront.

@eeyun commented on Fri Jun 23 2017

THIS SO MUCH. We've needed the ability to remove origins that have no packages for a while now. I think this request kind of goes hand in hand with #1051 . I squatted on origins at launch expected to be able to delete them and not been able to so I have like 10 that I don't need.

Aha! Link: https://chef.aha.io/features/APPDL-101