Table data extractor about scala-scraper HOT 12 CLOSED

It would be awesome if you shared your approach here, even if there was no interest now (which it seems there is), it would be a good resource for anyone dealing with this issue :) If the algorithm and data structure you chose to parse the table is general enough, I can surely help you implementing it in scala-scraper.

from scala-scraper.

Hi! Currently there is no specialized extractor for HTML tables. It would be a nice addition to have one, but does come with its challenges. For example, what data structure did you have in mind? As you mentioned in your example, the ability of cells to cover an arbitrary number of rows and columns can make the organization rather messy...

from scala-scraper.

I have something in mind - let me see if I can open a pull request in a few weeks. If I do end up coding something, where do you think it would show up in the codebase? In scalascraper/scraper/HtmlExtractor.scala?

from scala-scraper.

Yeah, I would expect it to be a new extractor in ContentExtractors. Looking forward to your pull request then!

from scala-scraper.

hi, any update with this ?

from scala-scraper.

from scala-scraper.

Hello samikrc ,
Thanks a lot for quick response and yes, I would be happy to check it, please do publish.

from scala-scraper.

Guys,

Sorry for the delay. Attaching two files, one containing the source code and the other containing some test code. Note that the test code is not automated - just some prints for manually checking if things look OK.

@ruippeixotog Saw your other email about the exciting features in the next version, including the "Content Extractors". This is probably too late to get included in that, but that is probably where this stuff can be integrated.

Ready to answer questions :-)

Thanks.
-Samik

TableExtractor.scala.txt
TableExtractorTester.scala.txt

from scala-scraper.

Also note that some of the methods are just stubs, but are easy to implement. Important methods are already implemented.

from scala-scraper.

Hi, any update on this? Did the code get used somewhere?

from scala-scraper.

Hi @samikrc, I ended up not using it anywhere for now - mostly due to my lack of time lately. I took a look at your code before and it seemed like a good implementation, it just needs to be converted to a more idiomatic extractor, like the regex extractors. I'll try to work on it in the next two weeks :)

from scala-scraper.

I have just added a new table content extractor to scala-scraper (e7d3fe6). I ended up writing the extractor from scratch, as it seemed easier for me to integrate it with the style of the other extractors this way.

Closing this now. If you find any bug with the implementation feel free to open another issue!

from scala-scraper.