dalehenrich / filetree Goto Github PK

When I copy a branched monticello package, the name of the MCPackage should be used to drive the name of the root directory ... keeping the branch in the package name is probably fine...

The methods visitClassTraitDefinition: visitScriptDefinition: visitTraitDefinition: in MCFileTreeAbstractStWriter are noops. We need to add support for these three methods in MCFileTreeStSnapshotWriter. We can safely ignore MCFileTreeStWriter since that is an older format ... the snapshot is the format of choice moving forward.

To read these definitions we'll need to extend the reader in MCFileTreeStSnapshotReader.

For each package in a file tree repo, from the image perspective, only the current commit is shown. Ideally, it would show all the versions in the repo, just like traditional mcz repos. I guess we'll need git integration for that, but I wanted note that it's missing.

CampSmalltalk#1 (comment)

failing build...

likely to be timeouts ... pharo build passes ... only recent changes were to builderCI framework and using new Metacello version

The Pharo Installation instructions advise to clone the filetree Git-repository and to create the MC-repository in a subdirectory of that Git instance. Since the filetree Git-repository is not actually required to use filetree, I suggest to delete the cloning step and replace the repository creation step with:

Gofer new
        repository: (MCFileTreeRepository new directory: 
                    (FileDirectory on: '/path/to/your/filetree/repository'));
        package: 'Your-Package';
        load.

Addtitionally, you might want to add that, instead of that last step, filetree repositories can be managed via the MC browser as usual.

I think that the monticello only files should have a .monticello extension ... I anticipate that there might be other meta-data files stashed around the package structure and putting extensions on all of the files/directories is probably a good idea...it's not like folks will every be typing these names...

There are two (three including issue #12) places in the package structure where there is potential duplication:

• method chunk
• class chunk

Method chunk

The method chunk specifies a class and method protocol (category). If the enclosing class directory is renamed or the class in the chunk is renamed, then there is an inconsistency that we can identify at the time that the definitions are created in the image.

If the method protocol starts with a * and does not match the current class (and the method is in a class directory), then we have another inconsistency

If the method protocol starts with a * and does not match the name of the current package (and the method is in the extensions directory) then we have an inconsistency

Class chunk

If the name of the class specified does not match the enclosing .class directory name that is an inconsistency

If the name of the category for the class does not match the enclosing package name that is an inconsistency..

If we throw an error when the inconsisties are identified during MC defintion creation , we can avoid much confusion later on.

The expected life cycle for packages stored in disk-based repositories (like git and svn) is:

1. aaa-dkh.1.mcz created as mcz file
2. aaa-dkh.2.mcz created as mcz file (ancestor aaa-dkh.1.mcz)
3. aaa-dkh.3.mcz created as mcz file (ancestor aaa-dkh.2.mcz)
4. aaa-dkh.3 copied to filetree repository (empty ancestry)
5. aaa-dkh.4 created in filetree repository (empty ancestry)
6. aaa-dalehenrichs.4 .mcz created as mcz file (ancestor aaa-dkh.3.mcz)

Now the situation exists where one would like to merge aaa-dalehenrichs.4 .mcz into aaa-dkh.4 in the filetree repository ...

In order for this to work i postulate that the following steps would work:

1. load aaa-dkh.4 into image
2. ADOPT aaa-dkh.3.mcz as ancestor (I think thats the sematics of adopt)
3. now merge aaa-dalehenrichs.4 .mcz and aaa-dkh.3.mcz should be the common ancestor

If these steps don't work we'll have to come up with something that will work ... otherwise we'll have to keep up-to-date ancestry information in the filetree repository which is not desirable...

Even though the mcz name of the package is not useful, it is useful to have some information about the version in the name ... the Monticello browser lists mcz version names so the filetree repository view should as well just to maintain consistency ...

Eventually we need to resolve the version history once and for all, but until then a little consistency is nice...

see Paul's suggested solution

Need to at least settle on a structure definition that is friendly towards traits and other (optional) extensions to the directory structure

in README.md, instructions is misspelled

Comparing a package in the image where the class has the method pharoMethod in a category to the package on disk in filetree (different version of repository) causes the meta data file for the method to be created:

GoferBar.tree/GoferBar/pharoMethod.meta

not sure why...

Happening in an experimental repository that may or may not survive ...

I think we have to take custom line endings into account.
might be that urgent for now, but most people will prefer using the system specific line-endings.

looks like .pkg was a bad choice for the "new" package directory ... probably need to come up with a different extension to oavoid confusion on Mac...

Suggestions?

Pharo and GemStone have different sort orders for strings (by default) and this affects the ordering of entries in the methodProperties.json file ... should use platform neutral sort order...

A workaround is suggested in the bug report...

Should use explicit extensions so that it is easy to add additional content to package structure and make it obvious to humans where the classes are ...

For a readme to show up nicely on github, it looks like we need to replace crs with lfs in comments ... I think that was already being done for the method source...

I think it is desirable to support multiple disk based package structures, so it would be prudent to have a marker file that is created in the repository that identifies the type of package stucture for the repo (basically identifying the reader/write to be used) ... one could make the case that this could even be on a package basis ... so it's prudent to figure this out

need to sort the keys to minimize spurious differences between property files written by different dialects

From Peter McLain:

The monticello tool gets an exception when trying to save changes to filetree://blah/blah/blah

Looks like a problem that Utf8 objects do not understand withUnixLineEndings (now that they are implemented as ByteArrays rather than CharacterSomethingOrOthers). The MCFileTreeStCypressWriter>>writeClassComment looks like:

writeClassComment: definition
fileStream nextPutAll: definition comment encodeAsUTF8 withUnixLineEndings

Stack from pharo/gemtools:

MessageNotUnderstood >> defaultAction (envId 0)
AbstractException >> _signalWith: (envId 0)
AbstractException >> signal (envId 0)
Object >> doesNotUnderstand: (envId 0)
Object >> _doesNotUnderstand:args:envId:reason: (envId 0)
MCFileTreeStCypressWriter >> writeClassComment: (envId 0)
[] in MCFileTreeStCypressWriter >> writeClassDefinition:to: (envId 0)
[] in MCFileTreeAbstractStWriter >> writeInDirectoryName:fileName:extension:visit: (envId 0)
[] in FileDirectory >> forceNewFileNamedo: (envId 0)
ExecBlock >> ensure: (envId 0)
FileDirectory >> forceNewFileNamedo: (envId 0)
MCFileTreeAbstractStWriter >> writeInDirectoryName:fileName:extension:visit: (envId 0)
MCFileTreeStCypressWriter >> writeClassDefinition:to: (envId 0)
[] in MCFileTreeStCypressWriter >> writeDefinitions: (envId 0)
Dictionary >> keysAndValuesDo: (envId 0)
MCFileTreeStCypressWriter >> writeDefinitions: (envId 0)
MCFileTreeWriter >> writeSnapshot: (envId 0)
MCMczWriter >> writeDefinitions: (envId 0)
MCMczWriter >> writeVersion: (envId 0)
MCFileTreeWriter >> writeVersion: (envId 0)
MCMczWriter class >> fileOut:on: (envId 0)
MCFileTreeRepository >> basicStoreVersion: (envId 0)
MCRepository >> storeVersion: (envId 0)
MCCmdSavePackage >> execute (envId 0)
[] in OBCommand >> clientExecute (envId 0)
OBUpdateConcentrator >> recordUpdatesDuring: (envId 0)
OBCommand >> clientExecute (envId 0)
GsNMethod class >> _gsReturnToC (envId 0)

see conversation starting with this post...

pharo-1.4 build

Occasionally I checkout different branches in a git repository from which I've loaded code into an image and it would be real nice to know the SHA of the commit that is loaded into the image

... when Issue 61 is fixed.

The workaround is in travisCI.st

I followed the installation instructions for Squeak4.3 in the README. During the last step I get the MNU shown below. Am I the only one?

Cheers,
Bernhard

MessageNotUnderstood: UndefinedObject>>dependencies
17 May 2012 6:40:01.152 pm

VM: Mac OS - Smalltalk
Image: Squeak4.3 [latest update: #12046]

SecurityManager state:
Restricted: false
FileAccess: true
SocketAccess: true
Working Dir /Users/bernhard/Documents/Squeak/4.4
Trusted Dir /foobar/tooBar/forSqueak/bogus/
Untrusted Dir /Users/bernhard/Library/Preferences/Croquet/Internet/Untrusted

UndefinedObject(Object)>>doesNotUnderstand: #dependencies
Receiver: nil
Arguments and temporary variables:
aMessage: dependencies
exception: MessageNotUnderstood: UndefinedObject>>dependencies
resumeValue: nil
Receiver's instance variables:
nil
GoferLoad>>addResolved:
Receiver: a GoferLoad
Arguments and temporary variables:
aResolvedReference: a GoferResolvedReference name: 'MonticelloFileTree-Core-dkh...etc...
version: nil
reference: nil
Receiver's instance variables:
gofer: a Gofer
model: a MCVersionLoader

[] in GoferLoad>>initializeOn:
Receiver: a GoferLoad
Arguments and temporary variables:
<
Receiver's instance variables:
gofer: a Gofer
model: a MCVersionLoader

[] in [] in Array(Collection)>>do:displayingProgress:every:
Receiver: {a GoferResolvedReference name: 'MonticelloFileTree-Core-dkh.61'}
Arguments and temporary variables:
<
Receiver's instance variables:
{a GoferResolvedReference name: 'MonticelloFileTree-Core-dkh.61'}
Array(SequenceableCollection)>>do:
Receiver: {a GoferResolvedReference name: 'MonticelloFileTree-Core-dkh.61'}
Arguments and temporary variables:
aBlock: [closure] in [] in Array(Collection)>>do:displayingProgress:every:
index: 1
indexLimiT: 1
Receiver's instance variables:
{a GoferResolvedReference name: 'MonticelloFileTree-Core-dkh.61'}
[] in Array(Collection)>>do:displayingProgress:every:
Receiver: {a GoferResolvedReference name: 'MonticelloFileTree-Core-dkh.61'}
Arguments and temporary variables:
<
Receiver's instance variables:
{a GoferResolvedReference name: 'MonticelloFileTree-Core-dkh.61'}
[] in [] in MorphicUIManager>>displayProgress:at:from:to:during:
Receiver: a MorphicUIManager
Arguments and temporary variables:
<
Receiver's instance variables:
toolBuilder: nil

BlockClosure>>on:do:
Receiver: [closure] in [] in MorphicUIManager>>displayProgress:at:from:to:during:
Arguments and temporary variables:
exception: ProgressNotification
handlerAction: [closure] in [] in MorphicUIManager>>displayProgress:at:from:to:...etc...
handlerActive: true
Receiver's instance variables:
outerContext: [] in MorphicUIManager>>displayProgress:at:from:to:during:
startpc: 86
numArgs: 0

[] in MorphicUIManager>>displayProgress:at:from:to:during:
Receiver: a MorphicUIManager
Arguments and temporary variables:
<
Receiver's instance variables:
toolBuilder: nil

BlockClosure>>ensure:
Receiver: [closure] in MorphicUIManager>>displayProgress:at:from:to:during:
Arguments and temporary variables:
aBlock: [closure] in MorphicUIManager>>displayProgress:at:from:to:during:
complete: nil
returnValue: nil
Receiver's instance variables:
outerContext: MorphicUIManager>>displayProgress:at:from:to:during:
startpc: 79
numArgs: 0

MorphicUIManager>>displayProgress:at:from:to:during:
Receiver: a MorphicUIManager
Arguments and temporary variables:
titleString: ''
aPoint: 1278@778
minVal: 0
maxVal: 1
workBlock: [closure] in Array(Collection)>>do:displayingProgress:every:
progress: [closure] in SystemProgressMorph>>position:label:min:max:
result: #(nil)
Receiver's instance variables:
toolBuilder: nil

ProgressInitiationException>>defaultResumeValue
Receiver: ProgressInitiationException:
Arguments and temporary variables:

Receiver's instance variables: 
    messageText:    nil
    tag:    nil
    signalContext:  ProgressInitiationException(Exception)>>signal
    handlerContext:     nil
    outerContext:   nil
    workBlock:  [closure] in Array(Collection)>>do:displayingProgress:every:
    maxVal:     1
    minVal:     0
    aPoint:     1278@778
    progressTitle:  ''

ProgressInitiationException(Exception)>>resume
Receiver: ProgressInitiationException:
Arguments and temporary variables:

Receiver's instance variables: 
    messageText:    nil
    tag:    nil
    signalContext:  ProgressInitiationException(Exception)>>signal
    handlerContext:     nil
    outerContext:   nil
    workBlock:  [closure] in Array(Collection)>>do:displayingProgress:every:
    maxVal:     1
    minVal:     0
    aPoint:     1278@778
    progressTitle:  ''

ProgressInitiationException>>defaultAction
Receiver: ProgressInitiationException:
Arguments and temporary variables:

Receiver's instance variables: 
    messageText:    nil
    tag:    nil
    signalContext:  ProgressInitiationException(Exception)>>signal
    handlerContext:     nil
    outerContext:   nil
    workBlock:  [closure] in Array(Collection)>>do:displayingProgress:every:
    maxVal:     1
    minVal:     0
    aPoint:     1278@778
    progressTitle:  ''

UndefinedObject>>handleSignal:
Receiver: nil
Arguments and temporary variables:
exception: ProgressInitiationException:
Receiver's instance variables:
nil
MethodContext(ContextPart)>>handleSignal:
Receiver: BlockClosure>>on:do:
Arguments and temporary variables:
exception: ProgressInitiationException:
val: nil
Receiver's instance variables:
sender: PasteUpMorph>>becomeActiveDuring:
pc: 17
stackp: 3
method: (BlockClosure>>#on:do: "a CompiledMethod(2372)")
closureOrNil: nil
receiver: [closure] in HandMorph>>sendFocusEvent:to:clear:

MethodContext(ContextPart)>>handleSignal:
Receiver: BlockClosure>>on:do:
Arguments and temporary variables:
exception: ProgressInitiationException:
val: nil
Receiver's instance variables:
sender: SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo:
pc: 17
stackp: 3
method: (BlockClosure>>#on:do: "a CompiledMethod(2372)")
closureOrNil: nil
receiver: [closure] in SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo:

ProgressInitiationException(Exception)>>signal
Receiver: ProgressInitiationException:
Arguments and temporary variables:

Receiver's instance variables: 
    messageText:    nil
    tag:    nil
    signalContext:  ProgressInitiationException(Exception)>>signal
    handlerContext:     nil
    outerContext:   nil
    workBlock:  [closure] in Array(Collection)>>do:displayingProgress:every:
    maxVal:     1
    minVal:     0
    aPoint:     1278@778
    progressTitle:  ''

ProgressInitiationException>>display:at:from:to:during:
Receiver: ProgressInitiationException:
Arguments and temporary variables:
argString: ''
argPoint: 1278@778
argMinVal: 0
argMaxVal: 1
argWorkBlock: [closure] in Array(Collection)>>do:displayingProgress:every:
Receiver's instance variables:
messageText: nil
tag: nil
signalContext: ProgressInitiationException(Exception)>>signal
handlerContext: nil
outerContext: nil
workBlock: [closure] in Array(Collection)>>do:displayingProgress:every:
maxVal: 1
minVal: 0
aPoint: 1278@778
progressTitle: ''

--- The full stack ---
UndefinedObject(Object)>>doesNotUnderstand: #dependencies
GoferLoad>>addResolved:
[] in GoferLoad>>initializeOn:
[] in [] in Array(Collection)>>do:displayingProgress:every:
Array(SequenceableCollection)>>do:
[] in Array(Collection)>>do:displayingProgress:every:
[] in [] in MorphicUIManager>>displayProgress:at:from:to:during:
BlockClosure>>on:do:
[] in MorphicUIManager>>displayProgress:at:from:to:during:
BlockClosure>>ensure:
MorphicUIManager>>displayProgress:at:from:to:during:
ProgressInitiationException>>defaultResumeValue
ProgressInitiationException(Exception)>>resume
ProgressInitiationException>>defaultAction
UndefinedObject>>handleSignal:
MethodContext(ContextPart)>>handleSignal:
MethodContext(ContextPart)>>handleSignal:
ProgressInitiationException(Exception)>>signal
ProgressInitiationException>>display:at:from:to:during:

ProgressInitiationException class>>display:at:from:to:during:
ByteString(String)>>displayProgressAt:from:to:during:
ByteString(String)>>displayProgressFrom:to:during:
Array(Collection)>>do:displayingProgress:every:
Array(Collection)>>do:displayingProgress:
GoferLoad>>initializeOn:
GoferLoad class(GoferOperation class)>>on:
Gofer>>execute:do:
Gofer>>execute:
Gofer>>load
UndefinedObject>>DoIt
Compiler>>evaluate:in:to:notifying:ifFail:logged:
[] in SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo:
BlockClosure>>on:do:
SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo:
SmalltalkEditor(TextEditor)>>evaluateSelection
SmalltalkEditor(TextEditor)>>doIt
SmalltalkEditor(TextEditor)>>doIt:
SmalltalkEditor(TextEditor)>>dispatchOnKeyboardEvent:
SmalltalkEditor(TextEditor)>>keyStroke:
[] in [] in TextMorphForEditView(TextMorph)>>keyStroke:
TextMorphForEditView(TextMorph)>>handleInteraction:fromEvent:
TextMorphForEditView>>handleInteraction:fromEvent:
[] in TextMorphForEditView(TextMorph)>>keyStroke:
StandardToolSet class>>codeCompletionAround:textMorph:keyStroke:
ToolSet class>>codeCompletionAround:textMorph:keyStroke:
TextMorphForEditView(TextMorph)>>keyStroke:
TextMorphForEditView>>keyStroke:
TextMorphForEditView(TextMorph)>>handleKeystroke:
KeyboardEvent>>sentTo:
TextMorphForEditView(Morph)>>handleEvent:
TextMorphForEditView(Morph)>>handleFocusEvent:
[] in HandMorph>>sendFocusEvent:to:clear:
BlockClosure>>on:do:
PasteUpMorph>>becomeActiveDuring:
HandMorph>>sendFocusEvent:to:clear:
HandMorph>>sendEvent:focus:clear:
HandMorph>>sendKeyboardEvent:
HandMorph>>handleEvent:
HandMorph>>processEvents
[] in WorldState>>doOneCycleNowFor:
Array(SequenceableCollection)>>do:
-- and more not shown --

the name of the .pkg directory should be sufficient for naming the package ... see Issue #10, as things get a bit complicated when writing new packages what happens if you have these 4 packages in a repository:

• sample-platform
• sample-platform.gemstone
• sample-platform.pharo
• sample-platform.squeak

I guess they are just fine and we can depend upon the .pkg directory to supply us with the package name, in which case Issue #10 must be clarified (I think Issue #10 is not an issue)

The version file without ancestry (see Issue 2 for ancestry disctission) really doesn' t have any useful information:

(name 'ConfigurationOfFileTree-dkh.9'
message '1.0 (dkh.9):

• repair package history'
  id '0b473e6e-6e60-4907-a4c7-afdbae7996ae'
  date '7 February 2012'
  time '6:45:31 pm'
  author 'dkh'
  ancestors ()

and just adds another file that needs to be dealt with on a merge operation.

The version comment (in my usage) is duplicated by the git commit comment.

The package name and id are somewhat useful but in a universe of git branches and forked repositories, I can't imagine that the package name and id are anything but misleading, especially in the absence of ancestry information ...

I am toying with the idea of creating the version file once when the mcz file is created (first load) and then "leaving it alone" ever afterwards ...

The name of the package in the image would be the base name: ''ConfigurationOfFileTree' in this case. The id can be safely ignored (I think) because I think it is only used one spot in the Monticello code base ... package equality(?) in the list of packages for a repository ... there aren't multiple versions of packages...

What we do for Issue 2 will involve the info in the version file, but commits to git repositories don't need to change any of this information.

several tests failing on squeak4.3 branch

see bug report for more detail.

see issue reported on Cypress issue list

I'm adding the Cuis cypress implementation source to the Cypress project project for interoperability testing and lo and behold, FileTree won't read the cypress format ...

presumably the list of issues for smallsource [1] the original fileTree project needs to be considered...

[1] https://github.com/finworks/smallsource

http://code.google.com/p/pharo/issues/detail?id=5772 completely removed FileDirectory from the image hence a rewrite of FileTree on top of FileSystem is needed in Pharo 2.0

• method meta information in separate method ... to make merging cleaner
• traits record
• others?

Dale,

When a package has required packages specified in Monticello, attempting to commit the package to a filetree repo fails.

What happens is that the dependencies directory in monticello.meta is removed during the commit (entire X.package directory is cleaned out to make room for the new version), and not recreated prior to attempting to forceNewFileNamed (sent from MCFileTreeWriter addString:at: to packageFileDirectory). The path is 'monitcello.meta/dependencies/Foo', but the system can't create the file in a non-existent directory.

For each dependent package, a file does not exist message is brought up. As a workaround, going into the parent package and adding the dependencies directory, then selecting 'create new file' does allow the write to finish.

depends upon dalehenrich/builderCI#13

see a1dfc01 for the initial commit ... need to double check whether tests were added along with the repository

The package properties should include a text encoding field ... I assume that the default encoding should be UTF8 ... which means that we need to explicitly encode/decode upon write/read in the proper encoding ..

I know this is a nasty issue with Monticello, so we shoul dnip it in the bud ....

When I first copied my package from a normal HTTP repository like SqueakSource, the files 'monticello.meta#package' and 'monticello.meta#version' were created on each XXXX.package directory. I think those files should be put on XXXX.package's monticello.meta sub directory. And correct file names are 'package' and 'version'.

MCFileTreeWriter>>addString:at: uses a hard-coded slash ('/'). I doubt it is a problem. (I'm using Pharo on Windows platform).

they should be if they are not

tests and core go hand in hand and splitting the tests into a separate repository is causing way too many headaches from a testing point of view ... simplify, simplify

When the RepositoryBrowser can't find the ancestor in the "current repository" it sets off on a journey searching through all repositories in RepositoryGroup class>>default for the matching package ... which can be excruciating ...

This situation typically occurs when you do a git checkout to a commit with a later version of the mcz package installed ...

For FileTree changes we should unconditionally compare with the single package present

when traits aren't present, FileTree should not add them to properties dictionary (with default values). Now that I'm porting to gemstone (that does not have traits) I've noticed that the traits entries cause unnecessary churn when alternately saving between GemStone and Pharo...

we need test cases where we use the full range of legal characters in both selectors and class names ... the $/ character is already escaped in method file names, but there may be additional characters that end up being invalid in file names on OSX and Linux

Right now in version 0.1 of the filetree format, I've got "seletor.st" files with the smalltalk source and "selector.meta" files with the author initials and timestamp ...

Of course the "selector.meta" files get modified whenever the source for a method is modified and the cost is one more file to monkey with on merge.

See [1] for additional discussions on the method meta data.

I split the meta data out separately so it would be easier to change the source of the meta data and I think I have a suggestion ...

My thought is that it is sufficient to use the package committers initials and package commit timestamp in the method annotations (my concerns[1] was that I'd have something unique, since I use that information when I'm switching method versions). Git can be used to get a clear picture of the method history (blame is pretty cool) so it's not necessary for Monticello to track the author history independently of git.

Otto points out that we could extract the actual author information from git to be displayed in the image, but correctly points out that this would add to the expense of creating the package data ...

[1] finworks/smallsource#12

'nuff said

I've left it running and tried twice in a clean and unmolested extent.

This command run in the stone's workspace never seems to get anywhere:

Gofer new
    repository: (MCFileTreeRepository new directory: 
                (FileDirectory on: '/opt/git/filetree/repository/'));
    package: 'MonticelloFileTree-Core';
    load.

I can put the extent I'm using up on dropbox if you want to try it yourself.

Probably should be using #contents in BinaryOrTextFile>>upToEnd

And by the way, all three method in BinaryOrTextFile should probably be pulled into the GLASS ...