Code Monkey home page Code Monkey logo

cpan2ebook's Introduction

What is this?

This is the source code of perlybook.org. The whole idea is to offer perl documentation in other formats. This allows others to read perl documentation on devices which are not capable of interpreting POD. Feel free to contact us, if you feel there should be another format added, especially if you are willing to contribute the code that provides it.

Setup

You can install dependencies via: cpanm -l local --installdeps .

cpan2ebook's People

Contributors

borisdaeppen avatar reneeb avatar rolpereira avatar stelim avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

cpan2ebook's Issues

Duplicate proposals for download filenames

The app proposes the same download filename for certain documents althoug they have a totally different content. For example the requests to convert the documents "perlunitut" and "perlreftut" both lead to the proposed download filename "perl-5.14.2.epub". This behaviour can be observed with all tutorials and most of the FAQ documents, for instance.

It seems to me, as if this could be a problem caused by the returning JSON data from MetaCPAN - or rather the app interpretating this data.

The request for "perlunitut" (http://api.metacpan.org//v0/search/autocomplete?&q=perlunitut) returns the following data (extract):

"fields" : {
  "documentation" : "perlunitut",
  "release" : "perl-5.14.2",
  "author" : "FLORA",
  "distribution" : "perl"
},

In contrast the request for "Data::Dumper" (http://api.metacpan.org//v0/search/autocomplete?&q=data++dumper) generates this response (extract)

"fields" : {
  "documentation" : "Data::Dumper",
  "release" : "Data-Dumper-2.131",
  "author" : "SMUELLER",
  "distribution" : "Data-Dumper"
},

I think the app uses the value of the key "release" as a part of the filename to send to the client. In the mentioned cases above, this unfortunately is ambiguous: There are several (tutorial) documents that have "perl-5.14.2" as their release.

Besides, this issue is an addition or special case to issue #12, because the content is not as expected as well.

Nevertheless a great app - please keep improving it!

Greetings,
rebos

Mojo setup

sounds dangerous:
[debug] Your secret passphrase needs to be changed!!!

I think this should be fixed sooner or later...

(Perltuts) Mobi version of تحليل-المواقع-باستخدام-lwp does not work

Problems تحليل-المواقع-باستخدام-lwp form Perltuts.

The EPUB version works fine, but the MOBI version contains the content of the início-rápido book.

I have seen the occurrence of the wrong name already at this line here: https://github.com/reneeb/cpan2ebook/blob/master/pod_book/lib/PodBook/Perltuts.pm#L59

A quick solution would be to not offer the link to the MOBI-Download at all.

Could be related with #25

Insert empty lines

If I might suggest something, then it would be to insert empty lines before titles. This would greatly enhance skimmability.
Disclaimer: Don't know if nor how this can be accomplished with epub.

android: can't download

What I do:
use perlybook.org from android device.
search for a module.
download as epub or mobi.

What I get:
"Download Manager" "Herunterladen nicht möglich, Inhalt nicht unterstützt".
in English:
"Download Manager" "Download not possible, content not supported".

User-IP seems to be tracked wrong (Starman as cause?)

On every user-request the IP is cached, and in case of to much requests per IP and timeinterval, the requests are not served - to protect the server from to much load.

But there seems to be something wrong in the code.
On the live system I can only find that the IP 127.0.0.1 is cached!
The following command shows this:

find tmp/Mojolicious-PodBook-Request/ | grep UID
    tmp/Mojolicious-PodBook-Request/f/1/UID+3a127+2e0+2e0+2e1.dat

THIS IS A SERIOUS BUG: because now the block is not per IP, but global!

The code for this is found on line 67 in lib/PodBook/CpanSearch.pm

65     my $remote_address;
66     my $pattern = $RE{net}{IPv4};
67     if ($self->tx->remote_address =~ m/^($pattern)$/) {
68         $remote_address = $1;
69     }

It looks like $self->tx->remote_address returns only the local loopback-IP.

Is this because of the use of Starman?

EPub utf-8 Problem

Input: EBook::MOBI
Type: EPub

A-Umlaut (ä) is not displayed correctly (author name).
In mobi there is no problem.

Replace CHI

Replace CHI with the wrapper Mojolicious::Plugin::CHI

Autocompletion does not work smooth

When I type "Mojolicious", there is only stuff like "Mojolicious::Renderer" proposed in Autocompletion, but not "Mojolicious" itself.

This was not so some time ago afaik.

use perlybook directly from kindle

The Kindle can browse perlybook.org directly, but download fails. It says that it can only dowload the format "MOBI" ... which perlybook offers. Maybe the file-ending needs to be uppercase? This would be stupid but worth a try...

Autocompletion

Autocompletion does not work any more. Is this a bug, or is it intended? (Or is it just my browser...)

error test message in logs

Since this date, those messages keep on appearing in the logs...

[Fri Sep 21 12:21:30 2012] [error] test

Seems not to make a lot of sense... maybe some lost/forgotten testing code?

Cache spamming in /tmp

Some code creates a lot of files in the root "/tmp".
I think the cause is in the Perltuts-caching-code.
Should be changed, atm there are over 2700 files there.
The caching-name and tmp-dir from the config should be used.

Nicer URL parameters

In some future releases the URL parameters could be changed so that they make more sense.

Now it is like this:

perlybook.org/?EPUB=EBUP&in_text=Module::Name
perlybook.org/?MOBI=MOBI&in_text=Module::Name

Better would be something like:

perlybook.org/?target=epub&source=Module::Name

Generated epub files seem to be not valid.

I can not open any generated EPub-File in iBooks (Mac OS X 10.9.5, iBooks 1.0.1 (281))

http://validator.idpf.org/ tells me, that there are errors in these files.
https://metacpan.org/pod/EBook::EPUB::Check provides a tool, to validate epub-Files:

➜  epub_check  local/bin/epubcheck ~/Downloads/Release_Catalyst-View-ByCode-0.25.epub
Epubcheck Version 3.0.1

Validating against EPUB version 2.0
WARNING: /Users/stefan/Downloads/Release_Catalyst-View-ByCode-0.25.epub/OPS/content.opf: spine contains only non-linear resources
WARNING: /Users/stefan/Downloads/Release_Catalyst-View-ByCode-0.25.epub/OPS/text/1.xhtml: Irregular DOCTYPE: found '-//W3C//DTD XHTML 1.0 Strict//EN', expecting '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">'.
WARNING: /Users/stefan/Downloads/Release_Catalyst-View-ByCode-0.25.epub/OPS/text/2.xhtml: Irregular DOCTYPE: found '-//W3C//DTD XHTML 1.0 Strict//EN', expecting '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">'.
ERROR: /Users/stefan/Downloads/Release_Catalyst-View-ByCode-0.25.epub/OPS/text/1.xhtml(62,20): 'app-': fragment identifier is not defined in 'OPS/text/1.xhtml'

Check finished with warnings or errors

TODO: Error Message from log (perl_mlb)

Should be checked...

[Mon May  7 09:47:22 2012] [info] eBook requested: perl_mlb
[Mon May  7 09:47:23 2012] [error] Failed to fetch 'http://api.metacpan.org/v0/source/ILYAZ/perl_mlb/MANIFEST': Not Found at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/EPublisher/Source/Plugin/MetaCPAN.pm line 44.

[Mon May  7 09:47:32 2012] [info] eBook requested: perl_mlb
[Mon May  7 09:47:32 2012] [error] Failed to fetch 'http://api.metacpan.org/v0/source/ILYAZ/perl_mlb/MANIFEST': Not Found at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/EPublisher/Source/Plugin/MetaCPAN.pm line 44.

About crashes

http://perlybook.org/about

Global symbol "$listsize" requires explicit package name at template "layouts/main.html.ep" line 14, near "".

14.     var listsize = <%= $listsize %>;

The "About" uses a different template but the Javascript-Code is in the main-template...

perl-5.14.2.epub crashes on Sony PRS-350

The book is quite huge (2,6M) and seems to have some errors, when you look inside the book.
E.g. the "EOB" thing in the text..?

Reader programs at the computer can read the book, but the EBook-Reader crashes when opening the TOC.
Would be interesting if other readers also crash...

Complains about unknown =begin sections

I get this message emitted with https://metacpan.org/source/KENTNL/Dist-Zilla-PluginBundle-Author-KENTNL-2.020001/lib/Dist/Zilla/PluginBundle/Author/KENTNL.pm#L908

POD ERRORS 
Hey! The above document had some coding errors, which are explained below: 
Around line 253: 
'=end MetaPOD::JSON v1.1.0' is invalid. (Stack: =begin MetaPOD::JSON) 

Its not actually invalid, that is the proper way to declare data sections in POD that are specific to only certain backends.

If you don't know what an =begin FOO is, then it should parse up to the subsequent =end FOO and ignore the hunk.

Add a Contributors Page

Once in a while we could add a Contributors-Page.
Akron could be listed there too: #27

No hurry, but a simple change that might motivate others to contribute.

Mojolicious.pm

22:29 nic ERROR: Module name not accepted.
22:29 garu really? I just typed 'Mojolicious' and it went ok
22:29 nic yeah, that's for Mojolicious.pm

From http://irclog.perlgeek.de/mojo/2012-04-29

This should be solved by delegating all input-checks to the metacpan-api.

Sending Mail fails

A try to send the Mobi for EBook::MOBI to my mail fails with this error message:



SMTP Failed to connect to mail server: Invalid argument
 at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Mojolicious/Plugin/Mail.pm line 55.
 at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Carp.pm line 100

95  

    # Icky backwards compatibility wrapper. :-(

96  

    local @CARP_NOT = $cgc ? $cgc->() : caller();

97  

    shortmess_heavy(@_);

98  

}

99  

100     

sub croak   { die shortmess @_ }

101     

sub confess { die longmess @_ }

102     

sub carp    { warn shortmess @_ }

103     

sub cluck   { warn longmess @_ }

104     

105     

BEGIN {

tap for more
Method:     

POST

URL:    

/

Base URL:   

http://perlybook.org

Parameters:     

{
  'book_selection' => 'distribution',
  'source' => 'EBook::MOBI',
  'target' => 'mobi',
  'always_mail' => '0',
  'mobi_send' => '[email protected]||never'
}

Stash:  

{
  'listsize' => 12,
  'appversion' => '0.22',
  'message' => '',
  'controller' => 'cpan_search',
  'via_mail' => 0,
  'optional_message' => '<br /><br /><br /><p><small>This website just came into the world. Please report any problems <a href=\'https://github.com/reneeb/cpan2ebook/issues\'>here</a>.</small></p>',
  'mobi_mail' => '',
  'action' => 'form',
  'mobi_sent' => 0,
  'config' => {
    'caching' => {
      'name_perltuts' => 'PerltutsCom',
      'seconds' => '864000',
      'name' => 'cpan2ebook_webservice'
    },
    'secret' => 'mypassphrase',
    'userblock_seconds' => '2',
    'optional_message' => '<br /><br /><br /><p><small>This website just came into the world. Please report any problems <a href=\'https://github.com/reneeb/cpan2ebook/issues\'>here</a>.</small></p>',
    'logging' => {
      'level' => 'info',
      'path' => {
        'general' => '/home/ebook/perlybook.log'
      }
    },
    'tmp_dir' => '/home/ebook/tmp',
    'exception' => {
      'to' => '[email protected]',
      'subject' => 'an error occured',
      'from' => '[email protected]'
    },
    'reverse_proxy' => '1',
    'mail' => {
      'howargs' => [
        'smtp.server.tld',
        'AuthUser',
        'smtpuser',
        'AuthPass',
        'smtppass'
      ],
      'from' => '[email protected]',
      'how' => 'smtp',
      'charset' => 'UTF-8',
      'encoding' => 'base64'
    },
    'autocompletion_size' => '12'
  }
}

Session:    

{}

Version:    

1.0

X-FORWARDED-HTTPS:  

0

User-Agent:     

Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0

Accept:     

text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language:    

de-de,de;q=0.8,en-us;q=0.5,en;q=0.3

Accept-Encoding:    

gzip, deflate

Content-Length:     

124

Host:   

perlybook.org

Content-Type:   

application/x-www-form-urlencoded

X-FORWARDED-FOR:    

79.216.185.115

REFERER:    

http://perlybook.org/

app.psgi

pod_pook/script/app.psgi is a copy of pod_pook/script/pod_book. Is there a reason why there need to be two copies of the same file? I would suggest to just use one file (or implement app.psgi as symlink, but this is not nice in git afaik)

Unprocessed Links in Mobi Format

In the CHI Mobi ebook I got from perlybook, there are many strings which seem like they ought to be links but are probably still in pod format, as such (I'm no expert on POD):

CHI::Driver::File|File

This pattern occurs in many places.

Template "not_found.development.html.ep" not found

If you activate the debug-mode in logging (config-file) there are quite a lot unpleasant messages produced for a normal request. Funny enough that a whole bunch of those messages come AFTER the delivery of the ebook.

[debug] POST / (Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:17.0) Gecko/17.0 Firefox/17.0).
[debug] Routing to controller "PodBook::CpanSearch" and action "form".
[info] Request from PodBook::CpanSearch=HASH(0x9efcbc0)->tx->remote_address, looking up 'EBook::MOBI', mapping to 'EBook-MOBI' fro 

[info] Sending for download: 'Release_EBook-MOBI-0.53.epub'
[debug] 200 OK (0.344557s, 2.902/s).
[debug] Rendering cached template cpan_search/form.html.ep.
[debug] Rendering cached template layouts/main.html.ep.
[debug] GET / (Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:17.0) Gecko/17.0 Firefox/17.0).
[debug] Routing to controller "PodBook::CpanSearch" and action "form".
[debug] Rendering cached template cpan_search/form.html.ep.
[debug] Rendering cached template layouts/main.html.ep.
[debug] 200 OK (0.010612s, 94.233/s).
[debug] GET /development-bundle/themes/ui-lightness/images/ui-bg_highlight-soft_100_eeeeee_1x100.png (Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:17.0) Gecko/17.0 Firefox/17.0).
[debug] Template "not_found.development.html.ep" not found.
[debug] Template "not_found.html.ep" not found.
[debug] Rendering cached inline template.
[debug] Rendering cached inline template.
[debug] 404 Not Found (0.024971s, 40.046/s).

This one is a bug I introduced recently, easy fix:

[info] Request from PodBook::CpanSearch=HASH(0x9efcbc0)->tx->remote_address, looking up 'EBook::MOBI', mapping to 'EBook-MOBI' fro

But no idea what this tells me:

[debug] Template "not_found.development.html.ep" not found.
[debug] Template "not_found.html.ep" not found.
[debug] Rendering cached inline template.
[debug] Rendering cached inline template.
[debug] 404 Not Found (0.024971s, 40.046/s).

Cache cleaning issue

Attribute (namespace) does not pass the type constraint because: Validation failed for 'Str' with value undef at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux/Moose/Meta/Attribute.pm line 1275.
Moose::Meta::Attribute::verify_against_type_constraint('Moose::Meta::Attribute=HASH(0x13f34690)', undef, 'instance', 'Moose::Meta::Class::ANON::SERIAL::1=HASH(0x13fd5088)') called at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux/Moose/Meta/Attribute.pm line 1262
Moose::Meta::Attribute::_coerce_and_verify('Moose::Meta::Attribute=HASH(0x13f34690)', undef, 'Moose::Meta::Class::ANON::SERIAL::1=HASH(0x13fd5088)') called at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux/Moose/Meta/Attribute.pm line 531
Moose::Meta::Attribute::initialize_instance_slot('Moose::Meta::Attribute=HASH(0x13f34690)', 'Moose::Meta::Instance=HASH(0x13013ec8)', 'Moose::Meta::Class::ANON::SERIAL::1=HASH(0x13fd5088)', 'HASH(0x13ca0e70)') called at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux/Class/MOP/Class.pm line 525
Class::MOP::Class::_construct_instance('Moose::Meta::Class=HASH(0x13cc2928)', 'HASH(0x13ca0e70)') called at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux/Class/MOP/Class.pm line 498
Class::MOP::Class::new_object('Moose::Meta::Class=HASH(0x13cc2928)', 'HASH(0x13ca0e70)') called at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux/Moose/Meta/Class.pm line 274
Moose::Meta::Class::new_object('Moose::Meta::Class=HASH(0x13cc2928)', 'HASH(0x13ca0e70)') called at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux/Moose/Object.pm line 28
Moose::Object::new('Moose::Meta::Class::ANON::SERIAL::1', 'chi_root_class', 'CHI', 'driver_class', 'CHI::Driver::File', 'root_dir', '/home/ebook/tmp', 'namespace', undef) called at constructor CHI::Driver::File::new (defined at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/CHI/Driver/File.pm line 30) line 4
CHI::Driver::File::new('Moose::Meta::Class::ANON::SERIAL::1', 'chi_root_class', 'CHI', 'driver_class', 'CHI::Driver::File', 'root_dir', '/home/ebook/tmp', 'namespace', undef) called at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/CHI.pm line 148
CHI::new('CHI', 'driver', 'File', 'root_dir', '/home/ebook/tmp', 'namespace', undef) called at /home/ebook/cpan2ebook/pod_book/script/cache_clean.pl line 30

Houskeeping is broken

The scripts executed with Cron throw errors or do not exists...

This means that for now the cache never gets cleaned. This is not a big issue, but once in a while it could be fixed...

Ebooks Don't Include Standalone Docs

I downloaded the ebook for Miyagawa's Starman. I noticed it only included the Pod for the modules but not for the CLI executable (AKA, 'launcher').

Example:

Modules

Plack::Handler::Starman - Plack adapter for Starman
Starman - High-performance preforking PSGI/Plack web server

Documentation

starman - Starman launcher

Items in the "Documentation" do not seem to be included.

Links to other pages...

Links in an ebook are not treated correctly. They point to the CPAN site, not to the other chapters...

Perltuts Encoding Problems

This URL gives an error: http://perlybook.org/perltuts?tutorial=%D0%B1%D0%B5%D0%B3%D0%BB%D0%BE%D0%B5-%D0%BE%D0%B7%D0%BD%D0%B0%D0%BA%D0%BE%D0%BC%D0%BB%D0%B5%D0%BD%D0%B8%D0%B5;format=epub

Perl dies with this message:

Cannot decode string with wide characters at /home/ebook/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2/x86_64-linux/Encode.pm line 174, <$in_fh> line 20.

The URL is the one called when you try to download "беглое-ознакомление" as EPUB.
MOBI has no problems, as far I can see.

Perltuts quick-start (epub) error

The URL

http://www.perlybook.org/perltuts?tutorial=quick-start;format=epub

gives the following error message in my browser:

XML-Verarbeitungsfehler: nicht wohlgeformt
Adresse: file:////home/bo/.mozilla/firefox/n226cysq.default/epub/6/OPS/text/1.xhtml
Zeile Nr. 276, Spalte 18: &&
-----------------^

Another URL works... e.g.:

http://www.perlybook.org/perltuts?tutorial=web-scraping-with-lwp;format=epub

So I think it is a specific bug of the book quick-start (epub) because also mobi works...

Don't upgrade Mojolicious

I have found some problems in the way I usually create downloads of data and thus this code here could be affected too.
This occurs since Mojolicious 3.63, I have asked for help here: mojolicious/mojo#428

According to cpan -D Mojolicious perlybook.org has the following version installed:

Installed: 3.43
CPAN:      3.67  Not up to date

So we should not upgrade systems libraries until we have an answer here, or tested on local machines.

Dists with no MANIFEST file create empty ebook

When request the epub for IO::Moose, an empty ebook is created. The logs say, that the distribution has no MANIFEST file.

TODO: Check if MetaCPAN provides a way to get a list of files included in the dist. If that is not possible, do not create an ebook but inform the user.

Problem restarting Server

When I am running

perl pod_book/script/start_stop_starman.pl

I get the message

sh: -c: line 0: syntax error near unexpected token `&&'
sh: -c: line 0: `&&  &&  nohup starman --listen :3030 --workers 10 --max-requests 200 --preload-app &'

This commit seemed to introduce the problem: 92649c0#pod_book/script/start_stop_starman.pl

This is urgent, since I made some updates and can't restart the site now.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.