Comments (8)
This is somewhat already supported. Any matched records (identified by fields
where rid="true") that are not explicitly bound to a class are skipped.
However, the fact that you must declare record mappings for ignored records is
cumbersome (but still recommended if you are at all worried about the integrity
of the file). It is also difficult to match records based on a field that does
NOT match a particular regex pattern or literal. So there are two things I'm
going to look into:
1. A stream level setting for ignoring unmatched records.
2. A field level setting to "inverse" the matched literal or regex pattern,
something like not="true".
And one point of clarification, filtering unmarshalled objects as opposed to
records does NOT increase memory requirements assuming you are processing the
file one record at a time (which is recommended for large files). A better
argument would be an increase in CPU utilization, although probably not
substantial.
Original comment by [email protected]
on 4 Jan 2013 at 2:45
- Added labels: Type-Enhancement
- Removed labels: ****
from beanio.
Original comment by [email protected]
on 4 Jan 2013 at 2:46
- Added labels: ****
- Removed labels: Type-Defect
from beanio.
Maybe you can expose an API for marshalling/unmarshlling to call a user-defined
callback. This way, filtering can be implemented, or encryption/decryption, or
other processing.
Original comment by [email protected]
on 23 Jan 2013 at 7:58
- Added labels: ****
- Removed labels: ****
from beanio.
Proceeding with functionality to ignore unidentified records for 2.0.4.
@cw10025 - Your request is kind of vague. Could you provide a sample callback
interface that you would like to see if you're still interested?
Original comment by [email protected]
on 2 Feb 2013 at 5:04
- Changed state: Started
- Added labels: Milestone-2.0.4
- Removed labels: ****
from beanio.
Thanks Kevin; we got it working by:
* Setting the Camel BeanIO Data Format to setUnexpectedRecords(true);
* Suppressing the SLF4J WARN logs for the BeanIODataFormat class.
Looking forward to 2.0.4 where we'll potentially be able to remove those
workarounds.
Original comment by [email protected]
on 22 Feb 2013 at 1:42
- Added labels: ****
- Removed labels: ****
from beanio.
Original comment by [email protected]
on 6 Mar 2013 at 3:13
- Changed state: Fixed
- Added labels: ****
- Removed labels: ****
from beanio.
I've upgraded to beanio 2.0.4 but am still seeing the WARN level messages. e.g.:
[main] WARN org.apache.camel.dataformat.beanio.BeanIODataFormat - BeanIO:
Unexpected record 'header' at line 56642: 498108 Alt Code ABCDEFG
Please consider reopening this issue. Thanks.
Original comment by [email protected]
on 12 Mar 2013 at 12:53
- Added labels: ****
- Removed labels: ****
from beanio.
Can you provide a mapping file and sample input that reproduces the issue?
Note that an unexpected record exception is typically a record out of order and
is not the same thing as an unidentified record.
Thanks,
Kevin
Original comment by [email protected]
on 12 Mar 2013 at 3:05
- Added labels: ****
- Removed labels: ****
from beanio.
Related Issues (20)
- Improvement for datetime parsinf in different timezones. HOT 2
- Change BeanReader and BeanWriter to implements Closeable HOT 2
- TypeConversionException and exception context
- @Group annotation ignores order flag HOT 2
- New instance of bean created everytime a segment is reused
- BeanIO reads preceding content when processing a segment HOT 2
- DateTypeHandlerSupport not thread-safe HOT 1
- JavaSE 8 - LocalDate & LocalTime Support HOT 3
- BeanIOFlatFileItemWriter - FooterCallback compatibility
- Want to format a field before calling fixed length validation on it HOT 3
- Automatic-Module-Name HOT 3
- CsvReader fails to handle a missing quotation character at the end of a line
- Example on writing a bean with annotations and no xml configuration to a file
- BeanIO fails to run under Wildfly 24 HOT 2
- Enhancement: Add support for (optional) delimiter for (repeated) groups/segments
- Repeating XML segments/groups/field causes Out Of Memory when type is set to none HOT 1
- @Field annotation don't have ignore property HOT 2
- Not working with graalvm/quarkus native? HOT 1
- Fixed length unmarshaller - Make it possible to customize calculating of record length HOT 7
- Fail to instantiate StreamFactory because of 'http://javax.xml.XMLConstants/property/accessExternalDTD' property is not recognized HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from beanio.