Code Monkey home page Code Monkey logo

ilias-downloader-cli's People

Contributors

dependabot[bot] avatar thetric avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

ilias-downloader-cli's Issues

No Courses found

Describe the bug
Hey, this is the first downloader that is able to work with the custom login setup of our ilias. However it is not able to finde any courses. Actually it is notclear if there are any courses found because there are unrecognized characters ("�[34m" and "�[0;39m") in the console output. However when trying do download all courses it just quits immediately.

To Reproduce
i just followed the console prompts

Logs
console output:

D:\Desktop>java -jar ilias-downloader-cli-4.0.4.jar -d ./ilias
�[34mSettings path: D:\Desktop\.\ilias\.ilias-downloader.json�[0;39m
Passwort von MeinemAccount:
�[34mLogging in at https://ilias.uni-freiburg.de/ilias.php?lang=en&client_id=unifreiburg&cmd=post&cmdClass=ilstartupgui&cmdNode=yc&baseClass=ilStartUpGUI&rtoken=�[0;39m
�[34mLogin at https://ilias.uni-freiburg.de/ilias.php?lang=en&client_id=unifreiburg&cmd=post&cmdClass=ilstartupgui&cmdNode=yc&baseClass=ilStartUpGUI&rtoken= succeeded �[0;39m
�[34mGet all courses and groups from https://ilias.uni-freiburg.de/ilias.php?baseClass=ilPersonalDesktopGUI&cmd=jumpToSelectedItems�[0;39m
�[34m>>> Gefundene Kurse:�[0;39m
Geben Sie die Nummern der zu synchronisierenden Kurse an (durch ein Leerzeichen getrennt) oder drücken Sie Enter, um direkt alle Kurse auszuwählen:
�[34mSyncing 0 courses: �[0;39m
�[34mSync gestartet�[0;39m
�[34mSync beendet�[0;39m

I cant create a log file - the argument is not recognized:

D:\Desktop>java -jar ilias-downloader-cli-4.0.4.jar -d ./ilias -c -Dlogback.configurationFile=logback-debug.xml
Ilias Downloader (CLI): unrecognized option '-D'

Environment (please complete the following information):

  • OS: Windows
  • Java Version (output of java -version):
    java version "13.0.1" 2019-10-15
    Java(TM) SE Runtime Environment (build 13.0.1+9)
    Java HotSpot(TM) 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
  • ilias-downloader-version: 4.0.4
  • Ilias Server URL: https://ilias.uni-freiburg.de/

Additional context
Add any other context about the problem here.

Exception when downloading from FH Dortmund ILIAS

The following exception occurs, when I try to download content from the ILIAS of the FH Dortmund:

Exception in thread "main" com.github.thetric.iliasdownloader.connector.domparser.impl.IliasHttpException: Failed to GET https://www.ilias.fh-dortmund.de/ilias/ilias.php?baseClass=ilPersonalDesktopGUI&cmd=jumpToSelectedItems: 
	at com.github.thetric.iliasdownloader.connector.domparser.impl.webclient.OkHttpIliasWebClient.checkResponse(OkHttpIliasWebClient.kt:94)
	at com.github.thetric.iliasdownloader.connector.domparser.impl.webclient.OkHttpIliasWebClient.getHtml(OkHttpIliasWebClient.kt:81)
	at com.github.thetric.iliasdownloader.connector.domparser.impl.course.CourseSyncServiceImpl.getHtml(CourseSyncServiceImpl.kt:38)
	at com.github.thetric.iliasdownloader.connector.domparser.impl.course.CourseSyncServiceImpl.connectAndGetDocument(CourseSyncServiceImpl.kt:33)
	at com.github.thetric.iliasdownloader.connector.domparser.impl.course.CourseSyncServiceImpl.getJoinedCourses(CourseSyncServiceImpl.kt:28)
	at com.github.thetric.iliasdownloader.connector.domparser.impl.WebIliasService.getJoinedCourses(WebIliasService.kt:30)
	at com.github.thetric.iliasdownloader.cli.UserPreferencesUpdateServiceImpl.updateSyncCourses(UserPreferencesUpdateServiceImpl.kt:38)
	at com.github.thetric.iliasdownloader.cli.UserPreferencesUpdateServiceImpl.updatePreferences(UserPreferencesUpdateServiceImpl.kt:27)
	at com.github.thetric.iliasdownloader.cli.CliController.startCliController(CliController.kt:36)
	at com.github.thetric.iliasdownloader.cli.CliKt.main(Cli.kt:36)

Fails to build with JDK 10

The master build fails when building with JDK 10 due to a Kotlin bug (see https://youtrack.jetbrains.com/issue/KT-23579). The JDK 8 build is not affected.

> Task :detektCheck FAILED
Exception in thread "main" java.lang.ExceptionInInitializerError
        at io.gitlab.arturbosch.detekt.core.KtCompiler.<init>(KtCompiler.kt:17)
        at io.gitlab.arturbosch.detekt.core.KtTreeCompiler$Companion.instance(KtTreeCompiler.kt:16)
        at io.gitlab.arturbosch.detekt.core.DetektFacade.<init>(DetektFacade.kt:21)
        at io.gitlab.arturbosch.detekt.core.DetektFacade$Companion.create(DetektFacade.kt:83)
        at io.gitlab.arturbosch.detekt.core.DetektFacade$Companion.create(DetektFacade.kt:69)
        at io.gitlab.arturbosch.detekt.cli.Runner.execute(Runner.kt:20)
        at io.gitlab.arturbosch.detekt.cli.Main.main(Main.kt:19)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 446
        at org.jetbrains.org.objectweb.asm.ClassReader.readUnsignedShort(ClassReader.java:2464)
        at org.jetbrains.org.objectweb.asm.ClassReader.readUTF8(ClassReader.java:2525)
        at org.jetbrains.org.objectweb.asm.ClassReader.readModule(ClassReader.java:761)
        at org.jetbrains.org.objectweb.asm.ClassReader.accept(ClassReader.java:646)
        at org.jetbrains.org.objectweb.asm.ClassReader.accept(ClassReader.java:507)
        at org.jetbrains.kotlin.resolve.jvm.modules.JavaModuleInfo$Companion.read(JavaModuleInfo.kt:67)
        at org.jetbrains.kotlin.cli.jvm.modules.CliJavaModuleFinder.findSystemModule(CliJavaModuleFinder.kt:44)
        at org.jetbrains.kotlin.cli.jvm.modules.CliJavaModuleFinder.access$findSystemModule(CliJavaModuleFinder.kt:25)
        at org.jetbrains.kotlin.cli.jvm.modules.CliJavaModuleFinder$systemModules$1.invoke(CliJavaModuleFinder.kt:37)
        at org.jetbrains.kotlin.cli.jvm.modules.CliJavaModuleFinder$systemModules$1.invoke(CliJavaModuleFinder.kt:25)
        at kotlin.sequences.TransformingSequence$iterator$1.next(Sequences.kt:149)
        at kotlin.sequences.FilteringSequence$iterator$1.calcNext(Sequences.kt:109)
        at kotlin.sequences.FilteringSequence$iterator$1.hasNext(Sequences.kt:133)
        at kotlin.sequences.FlatteningSequence$iterator$1.ensureItemIterator(Sequences.kt:254)
        at kotlin.sequences.FlatteningSequence$iterator$1.hasNext(Sequences.kt:241)
        at kotlin.sequences.SequencesKt___SequencesKt.none(_Sequences.kt:1239)
        at org.jetbrains.kotlin.cli.jvm.compiler.ClasspathRootsResolver.addModularRoots(ClasspathRootsResolver.kt:227)
        at org.jetbrains.kotlin.cli.jvm.compiler.ClasspathRootsResolver.computeRoots(ClasspathRootsResolver.kt:124)
        at org.jetbrains.kotlin.cli.jvm.compiler.ClasspathRootsResolver.convertClasspathRoots(ClasspathRootsResolver.kt:79)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment.<init>(KotlinCoreEnvironment.kt:230)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment.<init>(KotlinCoreEnvironment.kt:114)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.createForProduction(KotlinCoreEnvironment.kt:409)
        at io.gitlab.arturbosch.detekt.api.ProjectExtensionKt.createKotlinCoreEnvironment(ProjectExtension.kt:33)
        at io.gitlab.arturbosch.detekt.api.ProjectExtensionKt.<clinit>(ProjectExtension.kt:24)
        ... 7 more

ILIAS Login not working

My ILIAS Login is not working eventhough the password is fine.

Log:

Settings path: ./.ilias-downloader.json
XXXXXXs password:
Logging in at https://www.ilias.fh-dortmund.de/ilias/login.php
[main] ERROR c.g.t.i.c.CliController - Login failed
com.github.thetric.iliasdownloader.service.exception.IliasAuthenticationException: Login at https://www.ilias.fh-dortmund.de/ilias/login.php failed. Invalid credentials
at com.github.thetric.iliasdownloader.service.webparser.impl.webclient.OkHttpIliasWebClient.login(OkHttpIliasWebClient.kt:53) ~[ilias-downloader-cli-3.2.2.jar:na]
at com.github.thetric.iliasdownloader.service.webparser.impl.WebIliasService.login(WebIliasService.kt:25) ~[ilias-downloader-cli-3.2.2.jar:na]
at com.github.thetric.iliasdownloader.cli.LoginServiceImpl.createServiceFromConfig(LoginServiceImpl.kt:39) ~[ilias-downloader-cli-3.2.2.jar:na]
at com.github.thetric.iliasdownloader.cli.LoginServiceImpl.connect(LoginServiceImpl.kt:27) ~[ilias-downloader-cli-3.2.2.jar:na]
at com.github.thetric.iliasdownloader.cli.CliController.createIliasService(CliController.kt:67) ~[ilias-downloader-cli-3.2.2.jar:na]
at com.github.thetric.iliasdownloader.cli.CliController.startCliController(CliController.kt:28) ~[ilias-downloader-cli-3.2.2.jar:na]
at com.github.thetric.iliasdownloader.cli.CliKt.main(Cli.kt:36) ~[ilias-downloader-cli-3.2.2.jar:na]

The login at the given login URL with the same password works just fine.

Provide a native standalone executable for Linux

By using GraalVM's native-image tool we could provide users a standalone executable - without any Java dependencies except glibc.
Advantages:

  • lower memory footprint
  • faster startup

Drawbacks/open points:

  • larger app
  • currently no windows releases of Graal
  • worse (?) performance than Java
  • Mac builds?

File size download limit

As a user, I want to have the possiblity to restrict the download to items smaller than a certain size to exclude big archives/videos/... .

  • can set limit
  • limit setting is saved
  • sync does not download files larger than the limit (if present)

Crash on startup when -s option is omitted

Version:
1.0

Action:
java -jar ilias-downloader-cli-1.0.jar -d down

Expected:
The down/.ilias-downloader.json file is used for configuration, including the maximum file size.
The password prompt is then shown.

Actual:

Exception in thread "main" org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'null' with class 'null' to class 'long'. Try 'java.lang.Long' instead
	at org.codehaus.groovy.runtime.typehandling.DefaultTypeTransformation.castToNumber(DefaultTypeTransformation.java:163)
	at org.codehaus.groovy.runtime.typehandling.DefaultTypeTransformation.longUnbox(DefaultTypeTransformation.java:78)
	at com.github.thetric.iliasdownloader.cli.Cli.handleOptsReal(Cli.groovy:69)
	at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
	at com.github.thetric.iliasdownloader.cli.Cli.parseAndHandleOpts(Cli.groovy:39)
	at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:232)
	at com.github.thetric.iliasdownloader.cli.Cli.main(Cli.groovy:29)

max file size is reset to 0

Version:
1.0.1

Precondition:
initialized with ./idl -d down -s 64 -c
=> max file size in config file is 64

Action:
./idl -d down

Expected:
The max file size is still 64

Actual:
The max file size is reset to 0

Show a progress dialog when connecting to the server

Currently the user has no visual feedback whether the application is doing something or it has crashed. It would be helpful to show an indicator, e.g. a progress dialog.

Consider adding the indicator in these situtations:

  • between the initial server setup (for the server base url) and the login dialog
  • between the login dialog and the main ui

Course selection for sync

As an user, I want to choose the courses to be included in the sync to download only the contents I am interested in.

  • show available courses
  • edit selection
  • show my selection
  • only selected courses are synced
  • selection is persisted

one HTTP forbidden error causes program to abort

I don't know why, but ilias-downloader-cli can never seem to access this particular file:

Downloading file "Lösung-Bonusklausur2-MedStat-WS12-13.pdf" (16,500 bytes)... 
Exception in thread "main" com.github.thetric.iliasdownloader.service.exception.IliasException: Failed to GET https://www.ilias.fh-dortmund.de/ilias/webdav.php/ilias-fhdo/ref_9538//Alte_Klausuren//Miniklausuren//L%C3%B6sung-Bonusklausur2-MedStat-WS12-13.pdf: Forbidden
        at com.github.thetric.iliasdownloader.service.webparser.impl.webclient.OkHttpIliasWebClient.checkResponse(OkHttpIliasWebClient.kt:90)
        at com.github.thetric.iliasdownloader.service.webparser.impl.webclient.OkHttpIliasWebClient.getAsInputStream(OkHttpIliasWebClient.kt:84)
        at com.github.thetric.iliasdownloader.service.webparser.impl.WebIliasService.getContentAsStream(WebIliasService.kt:22)
        at com.github.thetric.iliasdownloader.cli.sync.ItemDownloadingItemVisitor.syncAndSaveFile(ItemDownloadingItemVisitor.kt:82)
        at com.github.thetric.iliasdownloader.cli.sync.ItemDownloadingItemVisitor.handleFile(ItemDownloadingItemVisitor.kt:48)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.walkIliasItemNode(CourseSyncServiceImpl.kt:107)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.visit(CourseSyncServiceImpl.kt:50)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.walkIliasItemNode(CourseSyncServiceImpl.kt:103)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.visit(CourseSyncServiceImpl.kt:50)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.walkIliasItemNode(CourseSyncServiceImpl.kt:103)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.visit(CourseSyncServiceImpl.kt:50)
        at com.github.thetric.iliasdownloader.service.webparser.impl.WebIliasService.visit(WebIliasService.kt:38)
        at com.github.thetric.iliasdownloader.cli.SyncController.executeSync(SyncController.kt:39)
        at com.github.thetric.iliasdownloader.cli.SyncController.startSync(SyncController.kt:28)
        at com.github.thetric.iliasdownloader.cli.CliController.startCliController(CliController.kt:41)
        at com.github.thetric.iliasdownloader.cli.CliKt.main(Cli.kt:30)

That wouldn't be so bad, especially since I encounter no trouble attempting to download this file manually through my browser. However, because it causes ilias-downloader-cli to abort, I am unable to finish syncing the other files in this course and all the remaining ones.

After downloading the file manually, I tried manually changing the timestamp of the file to trick ilias-downloader-cli into thinking it had already been synced, but that didn't work either. Actually, I noticed that all the timestamps of local synced files seem to be off +/- 5 minutes from the timestamps displayed in the Ilias web interface, but the difference appears to be somewhat random, so I can't guess the exact time ilias-downloader-cli expects the file to have and I am not sure whether it is comparing only minutes, or also seconds or even milliseconds, so I have given up on that workaround for now. I suppose I could dig into the sourcecode, but I am not that motivated and am hoping you guys can offer me an easier solution or workaround.

I would like if the program caught HTTP errors and prompted the user to abort, retry, or skip the current file. It would also be nice if there were an ignored files list in the configuration json. It would also be nice to figure out what causes the HTTP forbidden error, since this doesn't happen for me in my browser, so let me know if you have any ideas about how we could get to the bottom of that. Thanks.

Exception in thread "main" java.lang.NoSuchMethodError

after I selected I wanted to download all courses, the CLI crashed:

Exception in thread "main" java.lang.NoSuchMethodError: java.util.stream.Stream.iterate(Ljava/lang/Object;Ljava/util/function/Predicate;Ljava/util/function/UnaryOperator;)Ljava/util/stream/Stream;
        at com.github.thetric.iliasdownloader.cli.sync.ItemDownloadingItemVisitor.resolvePathOfParent(ItemDownloadingItemVisitor.kt:78)
        at com.github.thetric.iliasdownloader.cli.sync.ItemDownloadingItemVisitor.resolvePathAndCreateMissingDirs(ItemDownloadingItemVisitor.kt:71)
        at com.github.thetric.iliasdownloader.cli.sync.ItemDownloadingItemVisitor.handleFile(ItemDownloadingItemVisitor.kt:49)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.walkIliasItemNode(CourseSyncServiceImpl.kt:134)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.access$walkIliasItemNode(CourseSyncServiceImpl.kt:18)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl$visit$1.apply(CourseSyncServiceImpl.kt:56)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl$visit$1.apply(CourseSyncServiceImpl.kt:18)
        at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
        at java.util.ArrayList$ArrayListSpliterator.tryAdvance(Unknown Source)
        at java.util.stream.ReferencePipeline.forEachWithCancel(Unknown Source)
        at java.util.stream.AbstractPipeline.copyIntoWithCancel(Unknown Source)
        at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
        at java.util.stream.FindOps$FindOp.evaluateSequential(Unknown Source)
        at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.util.stream.ReferencePipeline.findFirst(Unknown Source)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.visit(CourseSyncServiceImpl.kt:63)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.walkIliasItemNode(CourseSyncServiceImpl.kt:130)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.access$walkIliasItemNode(CourseSyncServiceImpl.kt:18)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl$visit$1.apply(CourseSyncServiceImpl.kt:56)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl$visit$1.apply(CourseSyncServiceImpl.kt:18)
        at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
        at java.util.ArrayList$ArrayListSpliterator.tryAdvance(Unknown Source)
        at java.util.stream.ReferencePipeline.forEachWithCancel(Unknown Source)
        at java.util.stream.AbstractPipeline.copyIntoWithCancel(Unknown Source)
        at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
        at java.util.stream.FindOps$FindOp.evaluateSequential(Unknown Source)
        at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.util.stream.ReferencePipeline.findFirst(Unknown Source)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.visit(CourseSyncServiceImpl.kt:63)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.walkIliasItemNode(CourseSyncServiceImpl.kt:130)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.access$walkIliasItemNode(CourseSyncServiceImpl.kt:18)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl$visit$1.apply(CourseSyncServiceImpl.kt:56)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl$visit$1.apply(CourseSyncServiceImpl.kt:18)
        at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
        at java.util.ArrayList$ArrayListSpliterator.tryAdvance(Unknown Source)
        at java.util.stream.ReferencePipeline.forEachWithCancel(Unknown Source)
        at java.util.stream.AbstractPipeline.copyIntoWithCancel(Unknown Source)
        at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
        at java.util.stream.FindOps$FindOp.evaluateSequential(Unknown Source)
        at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.util.stream.ReferencePipeline.findFirst(Unknown Source)
        at com.github.thetric.iliasdownloader.service.webparser.impl.course.CourseSyncServiceImpl.visit(CourseSyncServiceImpl.kt:63)
        at com.github.thetric.iliasdownloader.service.webparser.impl.WebIliasService.visit(WebIliasService.kt:38)
        at com.github.thetric.iliasdownloader.cli.SyncController.executeSync(SyncController.kt:39)
        at com.github.thetric.iliasdownloader.cli.SyncController.startSync(SyncController.kt:28)
        at com.github.thetric.iliasdownloader.cli.CliController.startCliController(CliController.kt:41)
        at com.github.thetric.iliasdownloader.cli.CliKt.main(Cli.kt:30)

so I could login and even view all the subscribed courses.

my Java version:

java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)

I was running version 3.2.0 from the Github releases tab

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.