bryanwilhite / songhay.cloud.blobstorage Goto Github PK
View Code? Open in Web Editor NEWAzure BLOB Storage Repositories for JSON files in the Cloud
License: MIT License
Azure BLOB Storage Repositories for JSON files in the Cloud
License: MIT License
The AzureSearchMetadata
model should be marked obsolete because the base class has been modified with a Claims set that is redundant for what is defined in this model.
i may find anti-patterns in the need to write batch operations for azure storage inside of an application domain
az storage blob
[ π docs] might lead me toward a more minimalist, shell-script-based approach to interfacing with cloud storage
see Songhay.Affiliates
issue 3
because Azure.Storage.Blobs
(Nuget) is all in on System.Text.Json
β¦
https://github.com/BryanWilhite/SonghayCore/projects/1
the assumption today is that support for .NET 4.5 will be dropped (or the .NET 4.5 project will be dropped)
the other assumption is that there will be one .NET Standard project for this Solution, using <TargetFrameworks>
Using Azure Storage Emulator outside of the build server (VSTS) in the studio is a definitely a go. The catch is using it on a build server. There is a StackOverflow.com question from 2016 that addresses this issue.
here is a sketch:
internal async Task<IEnumerable<string>> GetYouTubeChannelBlobsAsync(string blobContainerChannelsPath, Func<IListBlobItem, bool> filter)
{
this.CheckProgramMetadata();
traceSource?.WriteLine($"{nameof(GetYouTubeChannelBlobsAsync)} getting channel BLOBs...");
if (filter == null) filter = i => true;
var blobs = await this._container
.ListBlobsAsync(useFlatBlobListing: true, pageLimit: 5, rootPath: blobContainerChannelsPath);
var tasks = blobs.Where(filter).Select(i => (i as ICloudBlob).ToStringFromBlobAsync());
await Task.WhenAll(tasks);
traceSource?.WriteLine($"{nameof(GetYouTubeChannelBlobsAsync)} downloaded {tasks.Count()} BLOBs.");
return tasks.Select(i => i.Result);
}
TODO item in StoreFeedsActivity.cs
in https://github.com/BryanWilhite/Songhay.Feeds
this error will be thrown:
System.ArgumentException: 'Listing snapshots is only supported in flat mode (no delimiter). Consider setting the useFlatBlobListing parameter to true.
Parameter name: blobListingDetails'
when this API call is made:
var blobs = await this._container.ListBlobsAsync(useFlatBlobListing: false, pageLimit: 5, rootPath: blobContainerChannelsPath);
it has not been released at the time of this issue inception which implies that i not plan to work the latest project for quite some time
i now see π why i added these methods in the heat of the moment:
these replace the need for these Repositories i was so enamored with
extensions based on these might be helpful:
internal async Task<JObject> DownloadBlobAsync(string blobName)
{
var blob = this._container.GetBlockBlobReference(blobName);
var json = await blob.ToStringFromBlobAsync();
return JObject.Parse(json);
}
internal async Task UploadBlobAsync(JObject jObject, string blobName)
{
var blob = this._container.GetBlockBlobReference(blobName);
blob.Properties.ContentType = MimeTypes.ApplicationJson;
traceSource?.TraceVerbose($"Uploading {blobName}...");
await blob.UploadTextAsync(jObject.ToString());
}
this might depend on completion of this: https://github.com/BryanWilhite/SonghayCore/projects/1
as of today, unit tests will fail because of lack of ordering
The TaggedJObjectTest suite is in need of some kind of way to express intent for an βorderedβ test. Until this MSTest2 issue is resolved this is the intent:
ShouldSaveTaggedJObjectToRepo()
ShouldLoadTaggedJObjectFromRepo()
ShouldDeleteTaggedJObjectInRepo()
continues on from BryanWilhite/SonghayCore#45
take learnings from Songhay.Blog
and centralize:
public static CloudStorageAccount ToCloudStorageAccount(this ConfigurationBuilder builder, string basePath, string connectionStringName)
{
if (builder == null) throw new NullReferenceException("The expected Configuration builder is not here.");
if (!Directory.Exists(basePath)) throw new DirectoryNotFoundException("The expected configuration base path is not here.");
builder
.SetBasePath(basePath)
.AddJsonFile(AppScalars.conventionalSettingsFile, optional: false, reloadOnChange: true);
var meta = new ProgramMetadata();
builder.Build().Bind(nameof(ProgramMetadata), meta);
if (meta.CloudStorageSet == null) throw new NullReferenceException("The expected cloud storage set is not here.");
var key = AppScalars.cloudStorageSetName;
var test = meta.CloudStorageSet.TryGetValue(key, out var set);
if (!test) throw new NullReferenceException($"The expected cloud storage set, {key}, is not here.");
if (!set.Any()) throw new NullReferenceException($"The expected cloud storage set items for {key} are not here.");
test = set.TryGetValue(connectionStringName, out var connectionString);
if (!test) throw new NullReferenceException($"The expected cloud storage set connection, {connectionStringName}, is not here.");
var cloudStorageAccount = CloudStorageAccount.Parse(connectionString);
return cloudStorageAccount;
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.