kichangkim / danboorudownloader Goto Github PK
View Code? Open in Web Editor NEWDanbooru image downloader.
License: MIT License
Danbooru image downloader.
License: MIT License
Sorry for the stupid question, but how to run it? What to install and where to enter the command:
> DanbooruDownloader dump MyDataset
hi
i'm receiving an error 403 sins today and i can't figure out why
this is the complet error
2023-02-06 00:42:00.1797|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-02-06 00:42:00.4405|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\DanbooruDownloader-master\DanbooruDownloader-master\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<b__0>d.MoveNext() in D:\DanbooruDownloader-master\DanbooruDownloader-master\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func1 job, Func
2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\DanbooruDownloader-master\DanbooruDownloader-master\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
i got this Problem for a day and i don't know from were it is
Dear Kim, it is very generous for you to share this dataset, but here i have a problem which i hope you can help me. for example, i want download 10k images whose tag is "upper body" from the Danbooru, how can i do to realize it?
I use a Linux system (Debian 12) and I can't seem to compile this program.
Is there a way to do that?
When I write DanbooruDownloader dump MyDataset
I see it:
2020-08-25 21:19:31.9651|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2020-08-25 21:19:33.3554|INFO|DanbooruDownloader.Commands.DumpCommand|Checking 200 posts ...
System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.) ---> System.NullReferenceException: Object reference not set to an instance of an object.
at DanbooruDownloader.Commands.DumpCommand.ConvertToPost(JObject jsonObject) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 240
at System.Linq.Enumerable.SelectArrayIterator`2.ToArray()
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at DanbooruDownloader.Commands.DumpCommand.Run(String path, Int64 startId, Boolean ignoreHashCheck, Boolean includeDeleted) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 66
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at DanbooruDownloader.Program.<>c__DisplayClass0_1.<Main>b__2() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Program.cs:line 46
at Microsoft.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
at DanbooruDownloader.Program.Main(String[] args) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Program.cs:line 59
---> (Inner Exception #0) System.NullReferenceException: Object reference not set to an instance of an object.
at DanbooruDownloader.Commands.DumpCommand.ConvertToPost(JObject jsonObject) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 240
at System.Linq.Enumerable.SelectArrayIterator`2.ToArray()
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at DanbooruDownloader.Commands.DumpCommand.Run(String path, Int64 startId, Boolean ignoreHashCheck, Boolean includeDeleted) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 66<--
My command: .\DanbooruDownloader.exe dump MyDataset --username ************ --api-key *********************
Log:
2023-06-16 17:49:51.7826|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:49:51.9206|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:49:54.9340|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:49:55.0212|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:49:58.0268|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:49:58.0767|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:50:01.0918|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:50:01.1784|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:50:04.1982|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:50:04.2422|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:50:07.2687|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:50:07.3574|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:50:10.3668|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:50:10.4347|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:50:13.4421|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:50:13.4887|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:50:16.4969|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:50:16.5789|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:50:19.5882|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:50:19.6706|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:50:22.6785|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:50:22.7251|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:53:11.0371|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:53:11.2264|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:53:14.2476|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:53:14.3370|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:53:17.3477|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:53:17.4175|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:53:20.4249|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:53:20.4696|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-06-16 17:53:23.4822|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2023-06-16 17:53:23.5674|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
2023-01-03 13:18:41.9115|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (5947490 ~ ) 2023-01-03 13:18:42.7698|ERROR|DanbooruDownloader.Commands.DumpCommand|System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 26 at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 49 --- End of stack trace from previous location --- at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func
1 job, Func2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Utilities\TaskUtility.cs:line 16
I'm trying to install this for sd-tagger-helper, and I cannot seem to figure it out. a step-by step process would be much appreciated!
C:\Users\Candy\Desktop\DanbooruDownloader>DanbooruDownloader dump MyDataset
2020-08-11 07:22:58.7893|INFO|DanbooruDownloader.Commands.DumpCommand|Downloading metadata ... (1 ~ )
2020-08-11 07:23:00.4392|INFO|DanbooruDownloader.Commands.DumpCommand|Checking 200 posts ...
System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.) ---> System.NullReferenceException: Object reference not set to an instance of an object. at DanbooruDownloader.Commands.DumpCommand.ConvertToPost(JObject jsonObject) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 240 at System.Linq.Enumerable.SelectArrayIterator`2.ToArray()
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at DanbooruDownloader.Commands.DumpCommand.Run(String path, Int64 startId, Boolean ignoreHashCheck, Boolean includeDeleted) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 66 --- End of inner exception stack trace --- at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at DanbooruDownloader.Program.<>c__DisplayClass0_1.<Main>b__2() in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Program.cs:line 46
at Microsoft.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args) at DanbooruDownloader.Program.Main(String[] args) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Program.cs:line 59---> (Inner Exception #0) System.NullReferenceException: Object reference not set to an instance of an object.
at DanbooruDownloader.Commands.DumpCommand.ConvertToPost(JObject jsonObject) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 240 at System.Linq.Enumerable.SelectArrayIterator`2.ToArray()
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) at DanbooruDownloader.Commands.DumpCommand.Run(String path, Int64 startId, Boolean ignoreHashCheck, Boolean includeDeleted) in D:\Projects\Machine Learning Projects\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 66<---
hello I appreciated your work thanks for sharing the tool.
I having a problem to store my images to sqlite. when I run the DanbooruDownloader.exe
my mydataset.sqlite still don have any data in it.
here is my database structure
dataset
├── danbooru.sqlite
└── images
├── 0
│ ├── danbooru_7235546_9fb231f273e0769e54013014d685f629.jpg
└── 1
├── danbooru_7235546_9fb231f273e0769e54013014d685f629.jpg
i did the cloudflaie update and it work for some time now i get this error and i cant get it to go
2023-02-17 11:32:35.6142|ERROR|DanbooruDownloader.Commands.DumpCommand|CloudflareSolverRe.Exceptions.CloudflareClearanceException: Clearance failed after 10 attempt(s).
at CloudflareSolverRe.ClearanceHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at DanbooruDownloader.Utilities.DanbooruUtility.GetPosts(Int64 startId, String username, String apikey) in D:\DanbooruDownloader-master\DanbooruDownloader\Utilities\DanbooruUtility.cs:line 34
at DanbooruDownloader.Commands.DumpCommand.<>c__DisplayClass1_1.<<Run>b__0>d.MoveNext() in D:\DanbooruDownloader-master\DanbooruDownloader\Commands\DumpCommand.cs:line 50
--- End of stack trace from previous location ---
at DanbooruDownloader.Utilities.TaskUtility.RunWithRetry(Func`1 job, Func`2 retryCondition, Int32 retryCount, Int32 retryDelay) in D:\DanbooruDownloader-master\DanbooruDownloader\Utilities\TaskUtility.cs:line 16```
Danbooru has a (new?) media_asset field that is unable to be written to the table as it isn't defined in the schema.
System.AggregateException: One or more errors occurred. (SQLite Error 1: 'table posts has no column named media_asset'.)
---> Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 1: 'table posts has no column named media_asset'.
at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
at Microsoft.Data.Sqlite.SqliteCommand.PrepareAndEnumerateStatements(Stopwatch timer)+MoveNext()
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader()
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
at DanbooruDownloader.Utilities.SQLiteUtility.InsertOrReplace(SqliteConnection connection, IEnumerable`1 posts) in I:\deepbooru\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs:line 116
at DanbooruDownloader.Commands.DumpCommand.Run(String path, Int64 startId, Int64 endId, Boolean ignoreHashCheck, Boolean includeDeleted, String username, String apikey) in I:\deepbooru\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs:line 227
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at DanbooruDownloader.Program.<>c__DisplayClass0_1.<Main>b__2() in I:\deepbooru\DanbooruDownloader\DanbooruDownloader\Program.cs:line 63
at Microsoft.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
at DanbooruDownloader.Program.Main(String[] args) in I:\deepbooru\DanbooruDownloader\DanbooruDownloader\Program.cs:line 78
Hi,
I love your project. Could you please provide some benchmark (accuracy, f1, etc) about the latest pretrained model in the release?
That will be a great help because I also want to train some models (EfficientNet, ViT, etc) by myself. By comparing the benchmark scores, I could understand the performance of the model better.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.