Comments (4)
It appears that the page between 20110827210108
and 20121007234751
wasn't modified then become a 404 in 20130110230837
. Wayback Machine will return the first version of the page it encountered if it wasn't modified after.
Ref: http://web.archive.org/cdx/search/cdx?url=http://tohokinemakan.jp/index.html&gzip=false
We do had a bug with the 404 of 2013. I've pushed a fix to 1.0.0
. If you update to 1.0.0
, it should show up with the -a/--all
flag.
➜ gem git:(master) ✗ bin/wayback_machine_downloader http://tohokinemakan.jp/index.html -l -a
[
{"file_url":"http://tohokinemakan.jp:80/index.html","timestamp":20130110230837,"file_id":"index.html"},
]
from wayback-machine-downloader.
I have upgraded to version 1.0.0. It seems that I need the --all
option for 200 pages as well.
For example: http://web.archive.org/cdx/search/cdx?url=http://tohokinemakan.jp/information.html&gzip=false
This page has 2 different version (with different content). Both of them have 200 response code. But without --all
the old one will be fetched.
$ wayback_machine_downloader http://tohokinemakan.jp/information.html -a -l
[
{"file_url":"http://tohokinemakan.jp:80/information.html","timestamp":20121008235908,"file_id":"information.html"},
]
$ wayback_machine_downloader http://tohokinemakan.jp/information.html -l
[
{"file_url":"http://tohokinemakan.jp:80/information.html","timestamp":20100815073403,"file_id":"information.html"},
]
from wayback-machine-downloader.
Try to update to 1.1.2
, it should download the correct last version of files with or without the --all
flag. It's also now capable of parsing more deeply snapshot pages if you have a large website.
from wayback-machine-downloader.
Fixed in 7eedc1a.
from wayback-machine-downloader.
Related Issues (20)
- # Failed to open TCP connection to web.archive.org:443 (Connection refused - connect(2) for "web.archive.org" port 443 HOT 1
- every index.html code brocken since today HOT 5
- error 503 stuff
- Am not a programmer but only a website designer for wordpress HOT 1
- 503 Service Temporarily Unavailable (OpenURI::HTTPError) HOT 3
- My Account Was Hacked Can Someone Help Me?
- Help please! port 443 HOT 26
- INSTRUCTIONS! Use this working version instead. HOT 8
- If THE GOD Gives Permission; Introduction Of A New Flag "-r, --requst_interval" Is Needed! Why? > Because .... : HOT 1
- Socket::ResolutionError) HOT 1
- Permission denied @ apply2files
- Only downloading index.html instead of whole website HOT 1
- Downloads everything but with 0 bytes
- Download specific files (pdf) contained in the entire Wayback Machine archive
- 504 Gateway Time-out (OpenURI::HTTPError) HOT 1
- Failed to open TCP connection to web.archive.org:443 (Connection refused - connect(2) for "web.archive.org" port 443) (Errno::ECONNREFUSED) HOT 2
- An existing connection was forcibly closed by the remote host. - SSL_connect (Errno::ECONNRESET)
- error when wayback_machine_downloader http://www.example.it --from 20180430143735
- Getting snapshot pages.C:/Ruby30-x64/lib/ruby/3.0.0/open-uri.rb:364:in `open_http': 504 Gateway Time-out (OpenURI::HTTPError) HOT 4
- Failed to open TCP connection to web.archive.org:443 (No connection could be made because the target machine actively refused it. - connect(2) for "web.archive.org" port 443) (Errno::ECONNREFUSED) HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wayback-machine-downloader.