Comments (8)
I've started work on this, in a separate branch ( json_format ). So far, I've written code to parse the JSON format into the ruby objects used internally by tosback. Please look it over, and see if that seems sensible and going in a good direction. Also, any advice about Ruby would be gratefully appreciated, as this is my first foray into the language.
from tosback2.
Oh great! Please have a look at the v1 data format: http://github.com/tosdr/tosdr.org/tree/dataspec
it's basically:
"fulltos": {
"tosback2": "a-tosback-sitename.tld",
"terms": {
"name": "Terms of Service", [equivalent of a current docname.name in the xml]
"url": "equivalent to the docname>url.name in the xml"
}
}
With @JimmStout we're also thinking about how we need to improve the current tosback xml structure to meet new challenges.
from tosback2.
I believe we just want to move the JSON services into our XML rules instead of modifying the script to use two sources. We need to verify that they aren't already in our rules though, and then begin to think of the best way to update the XML structure to support relationships between services.
from tosback2.
Hugo -- The link http://github.com/tosdr/tosdr.org/tree/dataspec is to a branch, not a particular file. What file were you referring to?
from tosback2.
Jimm -- so the primary format (for both tosback and tosdr) going forward is intended to be the XML format from Tosback2? In that case, I should modify the code in tosdr that currently reads the JSON to read the XML format instead, right? But, as you mentioned, the JSON format currently covers more cases than the XML format, so we can't simply replace the JSON format with the XML one, at least not yet.
So I'm not sure what the plan is, I think...
from tosback2.
Sorry, I was referring to tosdr.org/services/. Meanwhile, I wrote: https://github.com/tosdr/tosdr.org/wiki/Specification:-services
from tosback2.
Here's a list of the differences between the entries in the tosback2
files and in the tosdr/services files. First is listed the sites with
different URLs (the tosdr ones are listed first). Next is a list of the
sites that are only in the tosdr files, and not in tosback2 at all. I
didn't list the reverse, as there are hundreds of those. ;-) Finally is
a list of the 5 sites that are in sync, out of the total of 23 sites in
the tosdr list that have URLs listed.
amazon.com
Conditions of Use
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=508088
Privacy Notice
http://www.amazon.com/gp/help/customer/display.html/ref=footer_privacy?ie=UTF8&nodeId=468496
Security, Privacy & Accessibility
http://www.amazon.com/gp/help/customer/display.html/?nodeId=551434
AMAZON SILK TERMS & CONDITIONS
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=200775270
Amazon App Suite Legal Notices
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=201008370
Amazon Kindle Store Terms of Use
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=201014950
Amazon.com Privacy Notice
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=468496
Conditions of Use
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=508088
Interest-Based Ads http://www.amazon.com/b/?&node=5160028011
Kindle Cloud Reader Legal Notices
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=200701450
Kindle for Android Legal Notices
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=200495390
Kindle for Mac Legal Notices
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=200443370
Kindle for PC Legal Notices
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=200499360
Kindle for Windows 8 Legal Notices
http://www.amazon.com/gp/help/customer/display.html/?ie=UTF8&nodeId=200939440
Third Party Licenses
http://www.amazon.com/gp/help/customer/display.html/ref=help_search_1-1?ie=UTF8&nodeId=200910440&qid=1368379047&sr=1-1
apple.com
Privacy Policy http://www.apple.com/privacy/
iCloud Terms of Service http://www.apple.com/legal/icloud/en/terms.html
Privacy Policy http://www.apple.com/privacy/
Website Terms of Service http://www.apple.com/legal/terms/site.html
iCloud Terms of Service http://www.apple.com/legal/icloud/en/terms.html
iTunes Terms of Service http://www.apple.com/legal/itunes/us/terms.html
apple.com
Privacy Policy http://www.apple.com/privacy/
Website Terms of Service http://www.apple.com/legal/terms/site.html
iTunes Terms of Service http://www.apple.com/legal/itunes/us/terms.html
Privacy Policy http://www.apple.com/privacy/
Website Terms of Service http://www.apple.com/legal/terms/site.html
iCloud Terms of Service http://www.apple.com/legal/icloud/en/terms.html
iTunes Terms of Service http://www.apple.com/legal/itunes/us/terms.html
disqus.com
Privacy Policy
http://help.disqus.com/customer/portal/articles/466259-privacy-policy
Terms and Policies http://docs.disqus.com/kb/terms-and-policies/
Terms of Service
http://help.disqus.com/customer/portal/articles/466260-terms-of-service
Privacy Policy
http://help.disqus.com/customer/portal/articles/466259-privacy-policy
Terms of Service
http://help.disqus.com/customer/portal/articles/466260-terms-of-service
flickr.com
Privacy Policy
http://info.yahoo.com/privacy/us/yahoo/flickr/details.html
Yahoo! Terms of Service http://www.yahoo.com/r/ts
Privacy Policy http://info.yahoo.com/privacy/us/yahoo/
google.com
FAQ https://www.google.fr/intl/en/policies/faq/
Key terms https://www.google.fr/intl/en/policies/privacy/key-terms/
Privacy Policy https://www.google.com/intl/en/policies/privacy/
Terms of Service https://www.google.com/intl/en/policies/terms/
Google Analytics Terms of Service
https://www.google.com/intl/en/analytics/tos.html
Privacy Policy https://www.google.com/intl/en/policies/privacy/
Terms of Service https://www.google.com/intl/en/policies/terms/
skype.com
Business End User License Agreement
http://www.skype.com/intl/en-us/legal/business/eula/
Business End User License Agreement (US)
http://www.skype.com/intl/en-us/legal/business/eula-us/
Cookies Policy http://www.skype.com/intl/en-us/legal/privacy/general/#5
Etiquette http://www.skype.com/intl/en-us/legal/terms/etiquette/
Fair Usage Policy for Subscriptions
http://www.skype.com/intl/en-us/legal/terms/fair_usage/
Fair Usage Policy for US Minute Bundles
http://www.skype.com/intl/en-us/legal/terms/fair_usage_minute_bundles/
Group Video Calling Fair Usage Policy
http://www.skype.com/intl/en/legal/terms/gvc-fair-usage/
Privacy Policies http://www.skype.com/intl/en-us/legal/privacy/general/
Skype Emergency Calling
http://www.skype.com/intl/en-us/legal/skype-limited-emergency-calling/
Terms and Policies http://www.skype.com/intl/en-us/legal/
Terms of Service - Business
http://www.skype.com/intl/en-us/legal/business/terms/
Terms of Service - Business (US)
http://www.skype.com/intl/en-us/legal/business/terms-us/
Terms of Use http://www.skype.com/intl/en-us/legal/terms/tou/
Trademark Guidelines
http://www.skype.com/intl/en-us/legal/terms/trademark-guidelines/
Business End User License Agreement
http://www.skype.com/intl/en-us/legal/business/eula/
Business End User License Agreement (US)
http://www.skype.com/intl/en-us/legal/business/eula-us/
Etiquette http://www.skype.com/intl/en-us/legal/terms/etiquette/
Fair Usage Policy for Subscriptions
http://www.skype.com/intl/en-us/legal/terms/fair_usage/
Fair Usage Policy for US Minute Bundles
http://www.skype.com/intl/en-us/legal/terms/fair_usage_minute_bundles/
Group Video Calling Fair Usage Policy
http://www.skype.com/intl/en/legal/terms/gvc-fair-usage/
Privacy Policy English (International)
http://www.skype.com/intl/en/legal/privacy/general/
Skype Emergency Calling
http://www.skype.com/intl/en-us/legal/skype-limited-emergency-calling/
Terms of Service - Business
http://www.skype.com/intl/en-us/legal/business/terms/
Terms of Service - Business (US)
http://www.skype.com/intl/en-us/legal/business/terms-us/
Terms of Use http://www.skype.com/intl/en-us/legal/terms/tou/
Trademark Guidelines
http://www.skype.com/intl/en-us/legal/terms/trademark-guidelines/
tumblr.com
Privacy Policy http://www.tumblr.com/policy/en/terms_of_service
Terms of Service http://www.tumblr.com/policy/en/privacy
Privacy Policy http://www.tumblr.com/policy/en/privacy
Terms of Service http://www.tumblr.com/policy/en/terms_of_service
twitter.com
Privacy Policy https://twitter.com/privacy
Terms of Service https://twitter.com/tos
Privacy Policy http://twitter.com/privacy
Terms of Service http://twitter.com/tos
yahoo.com
Community Guidelines
http://docs.yahoo.com/info/guidelines/community.html
Privacy Policy http://info.yahoo.com/privacy/us/yahoo/details.html
Yahoo! Terms of Service http://www.yahoo.com/r/ts
Privacy Policy http://info.yahoo.com/privacy/us/yahoo/
Terms of Service http://info.yahoo.com/legal/us/yahoo/utos/terms/
youtube.com
Google Privacy Policy http://www.google.com/intl/en/policies/privacy/
Terms http://www.youtube.com/t/terms
Community Guidelines http://www.youtube.com/t/community_guidelines
Privacy Policy http://www.youtube.com/t/privacy_at_youtube
cloudant.com
FAQ https://www.cloudant.com/for-developers/faq/
Privacy https://www.cloudant.com/privacy/
Terms of Service https://www.cloudant.com/terms/
grammarly.com
Privacy Policy
http://privacy-policy.truste.com/verified-policy/www.grammarly.com
Terms of Service and License Agreement http://www.grammarly.com/terms
owncube.com
Privacy https://owncube.com/privacy.php
Terms https://owncube.com/terms.php
soundcloud.com
Community Guidelines http://soundcloud.com/community-guidelines
Privacy Policy http://soundcloud.com/pages/privacy
Terms of Service http://soundcloud.com/terms-of-use
spotify.com
Privacy Policy http://www.spotify.com/se/legal/privacy-policy
Terms and Conditions (Mobile)
http://www.spotify.com/se/legal/mobile-terms-and-conditions/
Terms and Conditions (Premium)
http://www.spotify.com/se/legal/premium-code-terms-and-conditions/
Terms and Conditions (Unlimited)
http://www.spotify.com/se/legal/unlimited-code-terms-and-conditions/
Terms and Conditions of Use
http://www.spotify.com/se/legal/end-user-agreement/
store.steampowered.com
Privacy Policy Agreement
http://store.steampowered.com/privacy_agreement/
Steam Subscriber Agreement
http://store.steampowered.com/subscriber_agreement/
xing.com
Privacy Policy http://www.xing.com/privacy
Terms http://www.xing.com/terms
500px.com
app.net
facebook.com
github.com
twitpic.com
from tosback2.
I would say most of the info from tosdr files is more recent than tosback… so…
from tosback2.
Related Issues (20)
- Feature: add OpenGraph-metatags to site header
- Add current policies for Facebook HOT 1
- Commit link error? HOT 2
- Muliple language versions may not be in sync :-( HOT 3
- TOSBack didn't grab new twitter.com update HOT 1
- Amazon commented out?
- Add discordapp.com HOT 1
- merge crawl folder into crawl_reviewed folder?
- Add cloudsight.ai
- deal with different web properties that share the same terms HOT 1
- ambiguous path i don't understand HOT 2
- Twitter docs seem to switch between GDPR and non-GDPR
- Upgrade Ruby version?
- Git quirk with Amazon Silk T&C HOT 3
- What is norecurse? HOT 1
- Wrong PP for nike.com
- tosback.org links HOT 1
- Issue with mysql connection on tosback.org server? HOT 4
- Crawler stopped (lack of inodes) HOT 6
- [FEEDBACK] Expanding the List of Sites : TOS.org
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tosback2.