Cross post from https://github.com/Microsoft/AzureNotebooks/issues/174

From python. repro:

with open('new_file2.txt', 'w+') as f:
    f.write('line1\nline2\n')
    f.seek(0)
    print("#1")
    text = f.read()
    print(text)
    print("#2")
    text = f.read()
    print(text)

Expect:

#1 
line1
line2
#2 

Actual:

#1
#2 
line1
line2

To repro:
Create a folder within mounted folder.
Write data to a file within that folder.
Attempt to du -h or ls -lah the folder. Size will be 0 always.

Attributes for size are likely not being populated or pulled down.

It is bad practice to call str.format() in a logger call because it's a waste of compute if the logger ever gets set so that debug calls are ignored. …

What you can do is in your logging.Formatter() call above is specify you want to use str.format()-style formatting. Then here you could do:
logger.debug("current max size {} is {}", path, max_size)

class WriteInfo(object): and class FileCache are both correct ways but we should standardize.

Trailing whitespace on line 310 (logger message)

Today executor = concurrent.futures.ThreadPoolExecutor(4) is static. This could be made to scale with CPU

We have return 'file' and return 'directory' bits in our code. We could use an enum there.

Currently, if you get a sas token from certain tools it comes with a '?' at the start, like a query string. Strip this at startup so that easy malformation doesn't cause failure.

To repro:
Pull a near-quota-sized ZIP into a mounted folder.
Expand this zip into the mounted folder.
No errors will be reported, but many files will be 0 bytes.

Potential fix:
Internally keep track of when a writer callback has failed, fail future writes immediately; prime this flag on first launch and refresh it occasionally.

1. the minimum level to log, should be reworded to add 'the available levels are'
2. add parens to if max_size < self.offset + data_length: and file_length < self.offset + data_length: to try and improve readablility (also just store the second half as a var and not calc twice :))
3. logger.warning('error writing ' + str(e)) could use an exception-logging call which would log the traceback
4. make if (not last.processing and (last.offset + len(last.data)) == offset and len(last.data) + len(data) < file.FileService.MAX_RANGE_SIZE): easier to read. by assigning some of these to variables

There is code that does

try:
    stuff
except KeyError:
    pass

If we just pass we can use contextlib.suppress()

Create a file that is 4k aligned
-rw-r--r-- 1 nbuser nbuser 110592 Nov 30 18:14 out.txt

try to cat it. and observe bad address.

The current REST API for Azure Files means that getting modified times for files is very expensive. Other Azure Storage APIs provide ways to get this along with file lists, allowing us to batch getting this data for multiple files all at once.

The code is commented out currently in getattr for doing this

Currently, there isn't an easy 'clear dir/file caches' when under test. Would be useful when doing multiple operations in a single area and modifying from outside the driver (for setup teardown mostly).

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request