Code Monkey home page Code Monkey logo

dataimporttool's Introduction

Data Import Tool

Join the chat at https://gitter.im/openMF/DataImportTool

Generating Excel Templates and Importing Bulk Data - Using simple servlet and Apache POI.

Build Status

Travis

Build Status

Setup Instructions

  1. Before you run the application, you need to have gradle installed and create a file dataimport.properties directly under your home directory. It should have the following 4 parameters:-

mifos.endpoint=https://demo.openmf.org/fineract-provider/api/v1/
mifos.user.id=mifos
mifos.password=password
mifos.tenant.id=default

sample file is at https://github.com/openMF/DataImportTool/blob/develop/dataimport.properties

  1. Use the command "gradle clean tomcatRunWar" to run the application and access it at localhost:8070/DataImportTool.

  2. If you are hosting the data import tool in the cloud, you need to ssh into the system to create the dataimport.properties file.

Note :- Default gradlew config will allow you to remote debug on port 8006.

Troubleshooting

  1. If you are hosting both this tool and the backend in the same system, you can change the debug port in gradlew.bat under mifosng-provider to listen in on a different port instead of 8005:- set DEFAULT_JVM_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,address=8006,server=y,suspend=n

  2. If you accidentally run out of heap size when running both in the same system, make sure your _JAVA_OPTIONS in Environment variables is set to -Xms512m -Xmx512m -XX:MaxPermSize=512m and it is getting picked up by gradle.

To Dos

  1. Transaction support for group loans and group savings -> (Blocker) Can't find an endpoint which returns all loan accounts or all loan accounts associated with groups. /loans returns only individual loans.
  2. Better workbook populator unit tests which will use FormulaEvaluator to evaluate if the data validation formulas and in-cell formulas embedded as Strings are not broken due to shifting of columns.
  3. Minor improvements to group related features once the release is stable (like sync repayments with meetings).

Dev Setup

  1. Eclipsify using command gradle clean cleanEclipse eclipse
  2. Import into project into workspace When opened, in the package explorer of the java perspective, right-click import->General->Existing projects into workspace. In the dialog that opens, specify the root directory option by browsing to and selecting the DataImportTool directory.

dataimporttool's People

Contributors

avikganguly01 avatar awesh avatar gitter-badger avatar nayan avatar shrunk7byadagi avatar vishwasbabu avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

nayan avikganguly01 prashanth5678 poojaconflux lingalamadhukar mujeeb1238 rubenyllanes awesh diunuge rkatsie abdul-qureshi rainerhessmer ricemutt vishwa1311 jogathogo shashvattrip swapnilchesa gabbyadjei sanjosesolutions myqyl4 kenngari prasadpatill2 musoni alvinlaustachanov binnygopinath sharathsankar nazeer1100126 audakel jroweboy gcjcse rajuan kwakutwumasi anuragmath gitter-badger skylabase basstone40 mexinad chris2001dc shahrahul1985 khetsothea siddalingk tintz21 colima agnessmeena awasum sampath7619 janiine anneren daveanol ashoklingala singoafrica brandhuge spranir francisguchie sportsbitenews lalchandmali nenge1 eadjeidjan makaraset maheshsundaramurthy hero78 bkartel1 amit-sr vganeshpokuru akingde stanacton shankha51 paulmsegeya bhanditz saltonmassally fordclifford rcyusa charulathakarthikeyan infracode-co darthmanwe kpassion kenlock ensemblelab yanjinlkham2458 apiatin kuvonchn

dataimporttool's Issues

Data Import Tool: characters are not reconsigned for provided office

Reported by Daniel at https://mifosforge.jira.com/browse/MIFOSX-2523
Original Description:

After I exported the clients for a new office, the following error appeared when I tried to download the Excel Sheet to import the groups:

IN THE DATA IMPORT TOOL PAGE:
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1

IN GIT BASH:
$ gradle clean tomcatrunwar
The ConfigurationContainer.add() method has been deprecated and is scheduled to be removed in Gradle 2.0. Please use the create() method instead.
The TaskContainer.add() method has been deprecated and is scheduled to be removed in Gradle 2.0. Please use the create() method instead.
:clean
:pmdMain
:compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
1 warning
:processResources
:classes
:war
:tomcatRunWar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/gradle-1.6/lib/logback-classic-1.0.9.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/Daniel/DataImportTool/build/tmp/tomcatRunWar/work/Tomcat/localhost/DataImportTool/WEB-INF/lib/logback-classic-1.0.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
Started Tomcat Server
The Server is running at http://localhost:8070/DataImportTool
08:41:47.313 [http-bio-8070-exec-3] INFO o.o.m.d.http.MifosRestClient - Status: 404
Servlet.service() for servlet [DataImportServlet] in context with path [/DataImportTool] threw exception
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
at com.google.gson.Gson.fromJson(Gson.java:803)
at com.google.gson.Gson.fromJson(Gson.java:768)
at com.google.gson.Gson.fromJson(Gson.java:717)
at com.google.gson.Gson.fromJson(Gson.java:689)
at org.openmf.mifos.dataimport.http.MifosRestClient.createAuthToken(MifosRestClient.java:116)
at org.openmf.mifos.dataimport.handler.client.GroupDataImportHandler.upload(GroupDataImportHandler.java:133)
at org.openmf.mifos.dataimport.web.DataImportServlet.parseAndUpload(DataImportServlet.java:64)
at org.openmf.mifos.dataimport.web.DataImportServlet.doPost(DataImportServlet.java:43)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1
at com.google.gson.stream.JsonReader.beginObject(JsonReader.java:374)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:165)

enable to proceed further

acer@COM1-PC ~/DataImportTool (master)
$ gradle clean tomcatRunwar

FAILURE: Build failed with an exception.

  • Where:
    Build file 'C:\Users\acer\DataImportTool\build.gradle' line: 15

  • What went wrong:
    A problem occurred evaluating root project 'DataImportTool'.

    Failed to apply plugin [id 'tomcat']
    java.lang.String cannot be cast to org.gradle.api.artifacts.Configuration

  • Try:
    Run with --stacktrace option to get the stack trace. Run with --info or --debug
    option to get more log output.

BUILD FAILED

Cant build the project

when i try to build with 'gradle clean tomcatRunWar' i get below error

FAILURE: Build failed with an exception.

  • Where:
    Build file '/home/ubuntu/DataImportTool/build.gradle' line: 17

  • What went wrong:
    A problem occurred evaluating root project 'DataImportTool'.

Could not find method create() for arguments [tomcatRun, class com.bmuschko.gradle.tomcat.tasks.TomcatRun, com.bmuschko.gradle.tomcat.TomcatPlugin$_configureTomcatRun_closure2@1afd72ef] on task set.

General Ledger

https://mifosforge.jira.com/browse/MIFOSX-2748
Original Description:
I understand that we can import the clients general ledger account balances and history - however - we cannot import the general ledger name. A big improvement and time saver for migrating to Mifos X would be the ability to import all general ledger account names under each major heading Asset - Liability - Equity - Income - Expense

Data Import Tool can't take Decimal Values

In Excel sheet, if you provide Principal or Interest as decimal values, for example, Principal value of 5000.30 and Interest value as 5.25%, It takes them as normal integers like 5000 and 5%.
Data import Tool excel sheet should support decimal numbers.

Increase timeout

For slow computers with low memory, we might need to increase the timeout from 100 secs to about 5 minutes to allow massive data to be imported.

Circular dependency error

Hi.

Can you help with the following error:

FAILURE: Build failed with an exception.

  • What went wrong:
    Circular dependency between the following tasks:
    :classes
    --- :compileJava
    --- :pmdMain
    --- :classes (*)

attach screenshot

best regards

toolerror

How to configure loan charges

We would like to configure a loan late penalty charge such that on every adversary of the loan repayment and it is late a penalty charge in charged on the spot as a % Of the amount due in that month as principal .

we have tried with all options but it is not working

unable to work on GIT shell 

**Warning**: git command could not be found. Please create an alias or add it to your PATH.

**Warning**: Could not find ssh-agent.

plz provide me with detailed procedure how to resolve this warning

Importing Loan charges using Data Import Tool

Reported by Daniel at https://mifosforge.jira.com/browse/MIFOSX-2187

Original Description:
I am exporting the loans with Data Import Tool, we have a charge called "death relief fund", it is about 60.00 lps per installment. How can I add this charge in the loan sheet using data import tool?
With existing Data import tool, I can only Add charge with charge time as "Specified due date". Other charge time is not yet supported.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.