Comments (3)
zhujiem
commented on July 4, 2024
Thanks for your suggestion! Any additional benefits to host datasets on huggingface?
from loghub.
davanstrien
commented on July 4, 2024
If you just shared the files via the Hub, there are a few main advantages:
- Git-based, so can fit into workflows more easily
- A nice [library](https://huggingface.co/docs/huggingface_hub/index to repos + files ) for programmatic uploading/downloading from the hub.
- local CDN for faster downloads
If you organize the dataset in a structured way, i.e. by language, you can also make it easy for people to download everything/download a specific subset of languages. The Stack dataset is a good example of this for a code dataset where you can either download everything or download only a particular language(s). Another way to achieve this would be via a dataset loading script.
A dataset loading script would give users even more control for what/how to download data. This requires a bit more work upfront but can be a nice way of avoiding users of the dataset reimplementing similar download scripts multiple times.
from loghub.
zhujiem
commented on July 4, 2024
Cool! If you are interested in help with this task, please go ahead. But please keep the README and LICENSE files.
from loghub.
Related Issues (20)
- How to generate log sequence matrix and KPI data from raw log data? HOT 1
- OpenStack Logs HOT 1
- Loghub Dataset 503 Service Unavailable HOT 1
- benchmark results for supervised anomaly detection HOT 1
- Can I redistribute some of the 2K sample logs? HOT 3
- Need a help to understand the HDFS2_ log file structure: 2016-01-13 07:48:28,240 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Successfully sent block report 0x13de8a8372744c, containing 1 storage report(s), of which we sent 1. The reports had 0 total blocks and used 1 RPC(s). This took 0 msec to generate and 1 msecs for RPC and NN processing. Got back one command: FinalizeCommand/5.
- Multi-class resources availability HOT 1
- authentication logs HOT 1
- 二维码过期 HOT 3
- Message types for these system logs HOT 1
- Anomaly Items in Dataset?
- Where is the complete HDFS.log raw file HOT 1
- 1
- 1
- How can i get the raw Spirit dataset HOT 1
- How to obtain the regular expressions used for various datasets HOT 1
- Can't get labeled HDFS logs, can't link to zenodo.org. HOT 1
- Hadoop log HOT 1
- Encoding issue with Linux log HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from loghub.