scraperwiki / code-scraper-in-browser-tool Goto Github PK

Should be in top level now, not in tool/

Same with the code!

Not sure how to hack share.js to send extra data, yet...

See comment in here:
https://github.com/frabcus/code-scraper-in-browser-tool/blob/master/http/index.html

For running code :) And perhaps for saving.

If I add

#!/usr/bin/tcc -r

then I get a complain from flock "flock: ./code/scraper: No such file or directory"

It is only if I use

#!/usr/bin/env tcc -r

that I get a useful error message

#!/usr/bin/env: tcc -r: No such file or directory

Not sure what feature to use in ACE.

Detect lines like this (Python):
File "./scraper", line 8

If it was loaded in parallel with the code.

Right now if you SSH and edit code/scraper, then next time someone goes to the code editor in the browser the ShareJS state will overwrite the changes the SSH person made.

Instead, it should notice that code/scraper has changed, perform a diff, create the relevant operational transforms and push them into the system.

See end of this thread:
https://groups.google.com/forum/#!searchin/sharejs/filesystem/sharejs/WIjZH_O34VU/uut-48zVWIcJ

Not sure why

So when one person hits "run" it starts polling output on the other

So you can't save before the load happens.

Would give some reassurance it is doing something.

Use case was I was running someone else's script that was busy doing lots of stuff using CPU, but was printing new output. Couldn't tell anything was happening.

It would be useful if it was possible to use this tool to write code that interacts with the output from another tool.

Currently within a dataset, if I use say, the twitter tool, it is not possible for me to use the editor to write code to interact with the database that that twitter tool created. I can ssh onto the box and so all that work there, and setup my cron job etc, but I'd rather not context switch from one thing to another unnecessarily.

If you keep reloading, you find newlines added to the end of file each time, because of the \n in the heredoc.

Maybe only add that \n if there isn't one at the end of the file already?

Kill the process - or else you can get stuck!

So that if they start doing other stuff it is in a directory on its own like a repository.

It would be helpful if a series of keystrokes would run the code (same as pressing "run").

c.f. codeacademy editor, where cmd (on mac) + enter = run

Hourly, daily, weekly, you choose!

Probably parses the crontab.

With git!

When using "Code in your browser," the editor is timing out when doing a relatively simple python operation.

The status bar next to "run" says saving at all times, and when pressing it, the editor freezes, and I get a 504 error at the top of the page.

The major problem here, is that I don't know whether it has saved or not before it freezes.

At least a link to the docs on Classic.

Bonus - copy all the docs over!

It doesn't feel like it is running without a spinner.

Or possibly it should go at end of console log?

A red flash with no text in it at the top of the page

Into one function

When you change the #! line

So can size console and editor window

Or can we find an ORM which constructs tables (and preferably has upserts) to use instead?

Make it save regularly - more often than just when you run.

Perhaps use a time after you type like Google Docs (used to) - wait 5 seconds then save.

Show when last saved somewhere.

How does this work with share.js?

It looks like it has broken and lost your work for a bit otherwise

Think lots of the timeout code could be simpler using them.

Failed only once; retrying worked just fine, without refresh.

Unknown new status! fatal: Unable to create '/home/code/.git/index.lock': File exists. If no other git process is currently running, this probably means a git process crashed in this repository earlier. Make sure no other git process is running and remove the file manually to continue. fatal: Unable to create '/home/code/.git/index.lock': File exists. If no other git process is currently running, this probably means a git process crashed in this repository earlier. Make sure no other git process is running and remove the file manually to continue. running

Rather than just .new

They both save a bit much

Using the various connection.on() events here, show in the autosave but (but in a more prominent colour) if the connection is broken

https://github.com/josephg/ShareJS/wiki/Client-API

As a Mac user I'd like one click whatever "Git clone or SSH in" does that copied the SSH command to the clipboard, so I can just paste it into Terminal.app

Windows users might want it to copy something else :)

Alternatively, is there an ssh:// URL type that is well supported we can use?

Perhaps by only grab new parts of logs/out. Can do with "head" easily I suspect

In the ShareJS server or on the filesystem?

I want it to be on the filesystem. It's not clear when or how to load from it though.

See this thread:
https://groups.google.com/forum/#!searchin/sharejs/set$20client/sharejs/WIjZH_O34VU/i82WpYG4FtMJ

exec endpoint doesn't support streaming properly yet.

Instead - perhaps spawn process in background and write to a log? Then poll the end of the log. This has advantage that it will combine cron executions with normal ones later.

To reduce round trips on startup

Does it reconnect? What happens then?

Getting "PHP Warning: Module 'rar' already loaded in Unknown on line 0" If running php scraper in browser.
Looks like problem described here: http://www.somacon.com/p520.php

After this, the php scraper does not work and ends with syntax error.
https://x.scraperwiki.com/dataset/gk4vsnq/settings