Comments (13)
The model for doing Windows and Linux tests is that you have multiple tests, typically parameterised (e.g. a pair of OS
and Test
), and some tests get run by one client, some by another. But having multiple clients should "just work", even if multiple clients can run a single test. In general, if a test T2
depends on a test T1
then all clients will separately do T1
- there is an assumption that the test T1
alters the local machine in some way that is required for T2
. But if T3
and T4
also depend on T1
, then both clients will do T1
, but then start doing different tests.
So generally speaking, throw multiple clients at it, and it should go faster - it's designed that way (and the multiple configurations thing is just a side-effect of that design, not the actual design itself).
from bake.
OK. But this means I have to allocate clients to some tests explicitly in the description of the tests...
from bake.
No requirement at all to allocated clients to tests - the Brains module does that all at all. The provide/require stuff is to limit decisions, if you leave it all out, any client can do anything.
from bake.
Yes, but then if I don't set explicit provide/require all clients will do everything all the time, which is not what I want. I guess what I want may be a different model.
from bake.
It won't aim for all clients to do everything - it aims to do everything once, but if a client is otherwise idle, it will do whatever it can, with the aim of completing everything as fast as it can. The clients will be busy all the time, but there will be no pointless duplication. If you can share some dependency information in your tasks, I could explain exactly what would happen with more clients.
from bake.
Sure, here is the main bake file:
type ImageId = String
type HostPort = Int
data Action = Compile
| Dependencies
| RunDocker ImageId HostPort
| Deploy
| IntegrationTest
| UITest
| EndToEndTest
deriving (Show,Read)
instance Stringy Action where
stringyTo = show
stringyFrom = read
patchFile :: String
patchFile = ".bake.patch"
imageName :: String
imageName = "capitalmatch/app"
main :: IO ()
main = do
putStrLn "Running CI"
let err = "You need to set an environment variable named $REPO for the Git repo"
repo <- fromMaybe (error err) `fmap` lookupEnv "REPO"
bake $
ovenNotifySlack "URI" "Channel" $
ovenIncremental $
-- what's this for ? Why the '=' sign...? looks like this is added to patch submitted
ovenPretty $
ovenStorePatch $
ovenGitNotes repo $
ovenGit repo "master" Nothing $
ovenNotifyStdout $
ovenTest (return allTests) execute
defaultOven{ovenServer=("127.0.0.1",5000)}
-- | Extract latest patch from state or patches informaiont
-- this introduces a constraint for type of patch and state to be equals, which works well
-- for git but might fail for other systems... Anyway, we don't care for now.
ovenStorePatch :: (Stringy patch, state ~ patch) => Oven state patch test -> Oven state patch test
ovenStorePatch oven@Oven{..} = oven { ovenPrepare = \ s ps -> do
let latest = extractLatestPatch s ps
writeFile patchFile latest
ovenPrepare s ps
}
extractLatestPatch :: (Stringy s) => s -> [s] -> String
extractLatestPatch s [] = stringyTo s
extractLatestPatch _ ps = stringyTo $ last ps
-- |Add ''notes'' containing result of build and attach it to head patch
-- The idea is to leverage standard git's notes mechanism to attach meta information from
-- build to commits thus allowing anybody to retrieve information about commits' builds
-- and actions that affected it.
ovenGitNotes :: (Stringy patch, state ~ patch) => String -> Oven state patch test -> Oven state patch test
ovenGitNotes repo oven@Oven{..} = oven{ ovenUpdate = \ up ps -> do
s <- ovenUpdate up ps
pushNotes repo up ps
return s
}
pushNotes :: (Stringy s) => String -> s -> [s] -> IO ()
pushNotes repo s ps = do
let latest = extractLatestPatch s ps
() <- cmd "git fetch" [repo] ["refs/notes/*:refs/notes/*"]
() <- cmd "git notes --ref=bake append" ["-m", "build successful", latest]
() <- cmd "git push" [repo] ["refs/notes/*"]
return ()
allTests :: [Action]
allTests = [ Compile
, Dependencies
, IntegrationTest
, UITest
, EndToEndTest
, Deploy
, RunDocker "app" 8080
, RunDocker "app-dev" 8081
]
execute :: Action -> TestInfo Action
execute Dependencies = run $ do
opt <- addPath ["."] []
() <- cmd opt "./build.sh images/ghc-clojure.uuid"
sleep 1
incrementalDone
execute Compile = depend [Dependencies] $ run $ do
opt <- addPath ["."] []
() <- cmd opt "./build.sh"
sleep 1
incrementalDone
execute IntegrationTest = depend [Compile] $ run $ do
opt <- addPath ["."] []
() <- cmd opt "./build.sh test"
incrementalDone
execute UITest = depend [Compile] $ run $ do
opt <- addPath ["."] []
() <- cmd opt "./build.sh ui-test"
incrementalDone
execute EndToEndTest = depend [Compile] $ run $ do
opt <- addPath ["."] []
() <- cmd opt "./build.sh end-to-end-tests"
incrementalDone
execute Deploy = depend [IntegrationTest, UITest, EndToEndTest] $ run $ do
patch <- readFile patchFile
-- we make sure that tag is not set to another image
() <- cmd $ "docker tag -f " ++ latest imageName ++ " " ++ tagged imageName patch
() <- cmd $ "docker push " ++ tagged imageName patch
incrementalDone
where
tagged name patch = name ++ ":" ++ patch
latest name = tagged name "latest"
execute (RunDocker image port) = depend [UITest, IntegrationTest, EndToEndTest] $ run $ do
Exit _ <- cmd $ "docker stop " ++ image
Exit _ <- cmd $ "docker rm " ++ image
() <- cmd "docker run" [ "--name=" ++ image
, "-d"
, "-p", show port ++ ":8080"
, "-v", "/home/build/data-" ++ image ++ ":/data"
, "capitalmatch/" ++ image ++ ":latest"
]
incrementalDone
from bake.
For info, cmd
now supports the AddPath
option, which means you can write cmd (AddPath ["."] []) ...
- which might be a bit cleaner.
So in your case, since you have a complete set of dependencies through the whole system, an extra client will hurt you and run all the tests twice. I think the problem is that we have depend
which requires that UITest
have been run on this client successfully before you continue. What you want is dependAnywhere
which requires the test to be finished, but not on this particular machine, since there is no side effect from UITest
, you just want to ensure it has passed. With that hypothetical and entirely reasonable addition, would you then see how things could work out? I suspect you would also want to tie the deployment to an individual client, so you don't accidentally deploy twice.
from bake.
Thanks, that's also what I inferred: adding another client won't help us as-is. And your proposal definitely makes sense. The model I have in mind is that of a DAG of "tests" that is projected on all available clients instances, possibly in parallel if possible.
from bake.
Yep, that's what we'd have if dependencies worked the way you wanted - I'll see what/when I can do that. Any indication how important it is for you?
from bake.
Between "Nice to have" and "Should have"... I would be happy to help if you can point me at the right direction :-)
from bake.
Brains is the beating heart and soul of Bake, it's also the most confusing bit. First thing is to plumb it through https://github.com/ndmitchell/bake/blob/master/src/Development/Bake/Core/Type.hs as an additional field. Once it reads https://github.com/ndmitchell/bake/blob/master/src/Development/Bake/Server/Brain.hs#L249-L307 you need to find the transitive closure of this new type of depends (just like I do for the existing depends), then add a filter in suitable
.
It might actually not be too hard, having looked at it - since the existing depends
is exactly the same for 90% of the places - but I wouldn't be shocked if it turned out to be deeper.
from bake.
To elaborate: UI Tests, integration tests and ETE Tests are an integral part of the deployment process and take a significant share of the time it takes to build our system. Here are the stats from bake:
Compile 47 5m14s 4h06m 31m35s
UITest 11 3m43s 40m57s 3m59s
EndToEndTest 46 2m48s 2h08m 6m21s
IntegrationTest 45 2m17s 1h43m 2m52s
Parallelizing execution of those tests would provide us on average a speed-up of 36% which is not negligible...
from bake.
Hmm, computation actually wrong as I do not take into account downstream (sequential) steps, but you get the idea :-)
from bake.
Related Issues (20)
- Support much bigger files HOT 1
- Notifications on clients coming/going are missing HOT 1
- Include patch author in the emails HOT 1
- Client crashes due to error 500 when finishing HOT 15
- Bake client process is too CPU heavy HOT 1
- Don't send emails until after updating the state
- Merged summary issues are in the wrong order
- Code is built twice HOT 6
- Build output disappears HOT 3
- Add -K1K to the test suite
- How do I use `bake gc` ? HOT 4
- GC should not remove data from latest incremental build
- Using 'Retry' in admin view adds a quote to patch number preventing it to be merged HOT 2
- Bring up clients as needed? (assuming clients are virtualized) HOT 7
- Problems building on ghc-7.8.4 HOT 6
- Document all functions HOT 4
- Build failure with extra-1.5.1 HOT 1
- Support multiple Git repositories from a single server? HOT 1
- bake-0.5 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bake.