odnoklassniki / one-nio Goto Github PK
View Code? Open in Web Editor NEWUnconventional I/O library for Java
License: Apache License 2.0
Unconventional I/O library for Java
License: Apache License 2.0
Добрый день!
Как реализовать в Router обработку http-запроса, в виде /service/1, /service/567565 (шаблон /service/{id})?
Есть аннотация @path, нужно ее использовать с указанием шаблона адреса?
Is there any tutorial 'hot to use serializer'?
I have serveral questions:
Thanks!
does one-nio framework has plans to support reactive-streams?
In the one.nio.pool.Pool, the object allocation and deallocation only occurs in the head of the pool. This way or strategy leads to other objects can't be checked in a pool heartbeat checker of application:
public final T borrowObject() throws PoolException, InterruptedException {
synchronized (this) {
for (long timeLimit = 0; ; ) {
// First try to get an idle object from the queue
T object = pollFirst();
if (object != null) {
return object;
}
public final void returnObject(T object) {
synchronized (this) {
if (!closed) {
if (waitingThreads > 0) notify();
addFirst(object); // Using addLast() for pool checker friendly
return;
}
}
destroyObject(object);
}
Suggest that Using addLast() instead of addFirst() in Pool.returnObject(), and no performance loss in LinkedList.
Serialization BigInteger with different versions of Java produces different result
byte[] res;
try (PersistStream stream = new PersistStream()) {
stream.writeObject(new BigInteger("123"));
res = stream.toByteArray();
}
System.out.println(res.length + " bytes, hash=" + Arrays.hashCode(res));
Java 8 output:
141 bytes, hash=-1637736666
Java 11 and 17 output:
169 bytes, hash=1514769574
If BigInteger serialized with Java 8 and deserialized with Java 11
try (DeserializeStream des = new DeserializeStream(res)) {
System.out.println(des.readObject());
}
Output:
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.bitCount
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.bitLength
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.lowestSetBit
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.firstNonzeroIntNum
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.bitCountPlusOne
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.bitLengthPlusOne
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.lowestSetBitPlusTwo
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.firstNonzeroIntNumPlusTwo
123
If vice versa
Output:
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.bitCountPlusOne
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.bitLengthPlusOne
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.lowestSetBitPlusTwo
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.firstNonzeroIntNumPlusTwo
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.bitCount
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.bitLength
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.lowestSetBit
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.firstNonzeroIntNum
123
byte[] res;
try (PersistStream stream = new PersistStream()) {
Instant now = Instant.now();
stream.writeObject(now);
stream.writeObject(now);
res = stream.toByteArray();
}
try (DeserializeStream des = new DeserializeStream(res)) {
System.out.println(des.readObject());
System.out.println(des.readObject());
}
Output:
2022-07-13T06:21:56.517Z
null
Hi!
I built my app with one-nio 1.2.0 and it doesn't start. In logs I can see:
/usr/bin/java: symbol lookup error: /tmp/libonenio.97307c0a.so: undefined symbol: OPENSSL_init_ssl
Both servers: where I built and where I tryed to start, have openssl version 1.0.2k-fips
What I should do?
Hello!
As I understand there's no support for HTTPS in current implementation of HttpClient
Maybe there's any way to handle it?
Thanks!
When I use RpcClient proxy service as a spring bean in spring boot, the java.lang.reflect.UndeclaredThrowableException is thrown in application bootstrap. A demo stack trace:
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy0.hashCode(Unknown Source)
at java.util.HashMap.hash(HashMap.java:339)
at java.util.HashMap.put(HashMap.java:612)
at EchoClient.main(EchoClient.java:57)
Caused by: one.nio.pool.PoolException: SocketPool[localhost:9000] createObject failed
at one.nio.pool.SocketPool.createObject(SocketPool.java:145)
at one.nio.pool.SocketPool.createObject(SocketPool.java:24)
at one.nio.pool.Pool.borrowObject(Pool.java:93)
at one.nio.rpc.RpcClient.invokeRaw(RpcClient.java:100)
at one.nio.rpc.RpcClient.invoke(RpcClient.java:43)
at one.nio.rpc.RpcClient.invoke(RpcClient.java:73)
... 4 more
Caused by: java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:118)
at one.nio.net.JavaSocket.connect(JavaSocket.java:63)
at one.nio.net.Socket.connect(Socket.java:76)
at one.nio.pool.SocketPool.createObject(SocketPool.java:135)
... 9 more
The demo likes this:
public static void main(String[] args) throws Exception {
Map<EchoService, EchoService> services = new HashMap<>();
ConnectionString conn = new ConnectionString(args[0]);
EchoService client = (EchoService) Proxy.newProxyInstance(
EchoClient.class.getClassLoader(),
new Class[] {EchoService.class},
new RpcClient(conn));
services.put(client, client); // UndeclaredThrowableException thrown here
// ... emitted
I want to create minimal auth. I want to check user ip in sever remote function, but I' don't understand how do this...
Hi There,
This project is still using an old implementation of LGTM's automated code review, which has now been disabled. To continue using automated code review, and receive checks on your Pull Requests, please install the GitHub App on this repository.
Thanks,
The LGTM Team
As in the title. We noticed that if the server returns some extra whitespace for transfer-encoding header this condition isn't met.
if (method != Request.METHOD_HEAD && mayHaveBody(response.getStatus())) {
if ("chunked".equalsIgnoreCase(response.getHeader("Transfer-Encoding: "))) {
response.setBody(readChunkedBody());
} else {
Deserialization BigDecimal with Java 1.8.0_351 does not work any more:
java.io.IOException: readObject() is not fully supported. See implementation notes.
at one.nio.serial.gen.NullObjectInputStream.unsupported(NullObjectInputStream.java:170)
at one.nio.serial.gen.NullObjectInputStream.readFields(NullObjectInputStream.java:50)
at java.math.BigDecimal.readObject(BigDecimal.java:3798)
at sun.reflect.Delegate14_BigDecimal.read(Unknown Source)
at one.nio.serial.GeneratedSerializer.read(GeneratedSerializer.java:127)
at one.nio.serial.DeserializeStream.readObject(DeserializeStream.java:75)
...
Привет!
Получаю ошибку при попытке десериализовать класс у которого есть поле типа ReentrantLock. Следующий код
DeserializeStream stream = new DeserializeStream( bytes );
return ( T )stream.readObject();
выбрасывает exception java.lang.IllegalAccessError: java/util/concurrent/locks/ReentrantLock$Sync
Hi folks. I am not sure if this is the right place to raise this but I thought I would create an issue none the less. Please feel free to move it to the appropriate place or close it out if this is not an issue.
I was attempting to use the HttpClient provided by one nio but I was unable to get it to connect to https://api.kucoin.com
. The client works with other servers that I have tried with but for this one server, it fails. I tried with both one-nio-1.5.0.jar and the latest one-nio-1.6.1.jar. For 1.5.0, I used openssl1.1 and for 1.6.1, I used openssl3.0. I cloned the repo and built it myself and ensured that the libonenio.so object exists in the jar and then compiled it with my client code shown below.
import one.nio.http.HttpClient;
import one.nio.http.HttpException;
import one.nio.net.ConnectionString;
import one.nio.pool.PoolException;
import java.io.IOException;
public class MyHttpClient {
public HttpClient client;
public MyHttpClient(String hostPort) {
this.client = new HttpClient(new ConnectionString(hostPort));
}
public void syncGet(String path) {
try {
System.out.println(this.client.get(path));
} catch (InterruptedException | PoolException | IOException | HttpException e) {
throw new RuntimeException(e);
}
}
}
However, I always ran into the following error when attempting to connect to the server:
Testing Kucoin connection
Exception in thread "main" java.lang.RuntimeException: one.nio.pool.PoolException: SocketPool[api.kucoin.com:443] createObject failed: javax.net.ssl.SSLException: error:0A000410:SSL routines::sslv3 alert handshake failure
at util.http.MyHttpClient.syncGet(MyHttpClient.java:24)
From the error message, it looks like the issue might be with sslv3. It's possible that the server here doesn't support sslv3 but my question is why is one.nio/openssl using SSLV3 to connect to the server? I would have thought that it would use TLSv1.2. I tried to forcefully disable sslv3 in but was still unsuccessful. I modified ssl.c as shown below.
https://wiki.openssl.org/index.php/List_of_SSL_OP_Flags
src/one/nio/net/native/ssl.c
JNIEXPORT jlong JNICALL
Java_one_nio_net_NativeSslContext_ctxNew(JNIEnv* env, jclass cls) {
AppData* appData = create_app_data();
if (appData == NULL) {
throw_by_name(env, "javax/net/ssl/SSLException", "Cannot allocate SSL app data");
return 0;
}
SSL_CTX* ctx = SSL_CTX_new(TLS_method());
if (ctx == NULL) {
free_app_data(appData);
throw_ssl_exception(env);
return 0;
}
SSL_CTX_set_mode(ctx, SSL_MODE_AUTO_RETRY | SSL_MODE_ENABLE_PARTIAL_WRITE | SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER);
fprintf(stderr, "\nSetting options to disable compression and ssl v3\n");
unsigned long currOptions = SSL_CTX_get_options(ctx);
fprintf(stderr, "\nCurrent options(pre set): %d\n", currOptions);
SSL_CTX_clear_options(ctx, currOptions); // Disable TLSv1.3 compat mode.
currOptions = SSL_CTX_get_options(ctx);
fprintf(stderr, "\nCurrent options(post clear): %d\n", currOptions);
currOptions |= SSL_OP_BIT(17); // Disable compression.
currOptions |= SSL_OP_BIT(25); // Disable SSL V3.
currOptions |= SSL_OP_BIT(29); // Disable TLSv1.3
SSL_CTX_set_options(ctx, currOptions);
fprintf(stderr, "\nCurrent options(post set): %d\n", SSL_CTX_get_options(ctx));
fprintf(stderr, "\nSuccessfully set options to disable compression and ssl v3\n");
// Disable read-ahead until premature socket close is fixed
// SSL_CTX_set_read_ahead(ctx, 1);
SSL_CTX_set_info_callback(ctx, ssl_info_callback);
SSL_CTX_set_app_data(ctx, appData);
setup_dh_params(ctx);
setup_ecdh_params(ctx);
return (jlong)(intptr_t)ctx;
}
The output looks as follows:
Testing Kucoin connection
Setting options to disable compression and ssl v3
Current options(pre set): 1179648
Current options(post clear): 0
Current options(post set): 570556416
Successfully set options to disable compression and ssl v3
Exception in thread "main" java.lang.RuntimeException: one.nio.pool.PoolException: SocketPool[api.kucoin.com:443] createObject failed: javax.net.ssl.SSLException: error:0A000410:SSL routines::sslv3 alert handshake failure
at util.http.MyHttpClient.syncGet(MyHttpClient.java:24)
570556416 === 0b100010000000100000000000000000
(17th, 25th and 29th bit are set)
[ubuntu@ip-10-20-82-90 ~]$ openssl s_client -connect api.kucoin.com:443 -tls1_2
CONNECTED(00000003)
depth=1 C = US, O = "Cloudflare, Inc.", CN = Cloudflare Inc ECC CA-3
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = kucoin.com
verify return:1
....
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2746 bytes and written 296 bytes
Verification error: unable to get local issuer certificate
---
New, TLSv1.2, Cipher is ECDHE-ECDSA-CHACHA20-POLY1305
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-ECDSA-CHACHA20-POLY1305
I would be really grateful if someone can help me out with this. Please do let me know if this is a known issue which has workarounds. I went through the issues posted previously but was unable to find any that matched this.
Do you plan to support java.util.Locale serialization? Or should I use my own Serializer<Locale>
?
Добрый день!
Планируется ли поддержка вэб-сокетов, как сервера так и клиента с возможностью реконнекта при сетевых проблемах?
In last rev:351d9d3490023c1c87d6d41ba1d5c03448ed055b
change field in @JsonName
, "name" to "value", but tests(JsonTest) not changed
Hi!
Create web app and use code:
@path("/organization")
public Response organization(Request request, HttpSession session) throws IOException {
Response response;
try {
if (request.getMethod() == Request.METHOD_GET) {
String name = request.getParameter("name");
String legalName = request.getParameter("legal_name");
String region = request.getParameter("region");
String locality = request.getParameter("locality");
String address = request.getParameter("address");
Method request.getParameter(String s) return wrong value.
Thanks.
This test is failed:
package one.nio.serial;
/*imports {...} */
public class SerializationTest extends TestCase {
/* others test-cases {...} */
static class SqlDatetime implements Serializable {
private final java.sql.Date date;
private final java.sql.Time time;
SqlDatetime(java.sql.Date date, java.sql.Time time) {
this.date = date;
this.time = time;
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
SqlDatetime sqlDatetime = (SqlDatetime) o;
return Objects.equals(date, sqlDatetime.date) &&
Objects.equals(time, sqlDatetime.time);
}
}
public void testSqlDatetime() throws IOException, ClassNotFoundException {
checkSerialize(new SqlDatetime(new java.sql.Date(System.currentTimeMillis()),
new java.sql.Time(System.currentTimeMillis())));
}
}
with an exception:
java.io.IOException: writeObject() is not fully supported. See implementation notes.
at one.nio.serial.gen.NullObjectOutputStream.unsupported(NullObjectOutputStream.java:159)
at one.nio.serial.gen.NullObjectOutputStream.writeLong(NullObjectOutputStream.java:130)
at java.util.Date.writeObject(Date.java:1323)
at sun.reflect.Delegate1_Date.calcSize(Unknown Source)
at one.nio.serial.GeneratedSerializer.calcSize(GeneratedSerializer.java:115)
at one.nio.serial.CalcSizeStream.writeObject(CalcSizeStream.java:122)
at sun.reflect.Delegate0_SqlDatetime.calcSize(Unknown Source)
at one.nio.serial.GeneratedSerializer.calcSize(GeneratedSerializer.java:115)
at one.nio.serial.CalcSizeStream.writeObject(CalcSizeStream.java:122)
at one.nio.serial.SerializationTest.clone(SerializationTest.java:38)
at one.nio.serial.SerializationTest.checkSerialize(SerializationTest.java:84)
at one.nio.serial.SerializationTest.testSqlDatetime(SerializationTest.java:391)
That's because one-nio generates Serializer like this (decompile to source):
public final class Delegate6_Date extends MagicAccessorImpl implements Delegate {
public Delegate6_Date() {
}
public final void calcSize(Object var1, CalcSizeStream var2) throws IOException {
((Date)var1).writeObject(NullObjectOutputStream.INSTANCE);
}
public final void write(Object var1, DataStream var2) throws IOException {
((Date)var1).writeObject(NullObjectOutputStream.INSTANCE);
}
public final Object read(DataStream var1) throws IOException, ClassNotFoundException {
java.sql.Date var2;
var1.register(var2 = new java.sql.Date);
var2.readObject(NullObjectInputStream.INSTANCE);
return var2;
}
}
and NullObjectOutputStream throws this exception.
At the moment the read
method of for example LocalDate
is generated to something like this
public final Object read(DataStream var1) throws IOException, ClassNotFoundException {
LocalDate var2;
var1.register(var2 = new LocalDate);
JavaInternals.unsafe.putInt(var2, 12L, var1.readInt());
JavaInternals.unsafe.putShort(var2, 16L, var1.readShort());
JavaInternals.unsafe.putShort(var2, 18L, var1.readShort());
var2.readObject(NullObjectInputStream.INSTANCE);
return var2;
}
But the LocalDate#readObject
will throw InvalidObjectException
because it uses serialization proxy java.time.Ser
. The simple solution is to exclude calling of this method from generated code like this:
Repository.setOptions(LocalDate.class, Repository.SKIP_READ_OBJECT);
// ... same for all java.time.* classes
However it could be made in a general way if we check for presence of writeReplace
method and exclude readObject
automatically.
Do you have plans for this? I suppose I can make a PR, it looks like a simple condition in DelegateGenerator#generateRead
.
Hi!
Java 17 is supported?
How run to java 17?
I have a field defined as String id. Most of the time it is a number but sometimes it is a String. The received json is sometimes formatted as number "id":123 and sometimes as string "id":"foo". Serialization fails as it expects a String not a number.
How do I handle this case in code? I'd like the number to be coverted to String. Do I use a custom @SerializeWith?
There are no sources/instructions how to build it.
Hello,
Is there any way to make requests with http client and specify different timeout for each request?
Thanks!
I want to add a user login check handler in my application when using the HttpServer, but I can't find a graceful way to do this in one.nio.http package.
For more convinient to use the serial package will be nise when library generate code for some type to generate code for all types of fields of this class.
Example:
I have class
class MyDto {
private BigDecimal decimal;
}
and 2 JVMs between which will be data exchange. (First - serialize, second - deserialize)
Even if I preregister this class on both JVM like
Repository.preload(MyDto.class);
second jvm will not be able to deserialize a data because there is no a serializer for BigDecimal
.
I know that we can request a schema but it's no needed here we have one already.
The java.nio.channels.Selector has a wakeup method so that externally the thread waiting on the selector can be notified. E.g. it can be used to signal the io thread to start sending some payload over the network.
The one.nio.net.Selector doesn't have a wakeup. This means that one-nio can't be used as a replacement for networking code that relies on the wakeup.
Hi!
What about to add this package to Maven Central and/or to JCenter?
I am doing a course project in the project's boundaries "Technopolis" (https://github.com/kilinochi/2019-highload-dht) and I have a problem with server - he is not running.
My main class
` public final class Server {
private static final int PORT = 8080
private Server() {
// Not instantiable
}
public static void main(String[] args) throws IOException {
// Temporary storage in the file system
final File data = Files.createTempDirectory();
// Start the service
final DAO dao = DAOFactory.create(data);
final Service service =
ServiceFactory.create(
PORT,
dao);
service.start();
Runtime.getRuntime().addShutdownHook(
new Thread(() -> {
service.stop();
try {
dao.close();
} catch (IOException e) {
throw new RuntimeException("Can't close dao", e);
}
}));
}
}`
and RestService class, which implement Service
`public final class RestService extends HttpServer implements Service {
private static final Logger logger = LoggerFactory.getLogger(RestService.class);
private static final String API = "/v0/entity";
private final DAO dao;
public RestService(final int port, @NotNull final DAO dao) throws IOException {
super(getConfig(port));
logger.info("Server is running on port " + port);
this.dao = dao;
}
@Override
public void start() {
}
@Override
public void stop() {
}
@Path(API)
public Response entity (@Param("id") final String id, final Request request) {
if(id == null || id.isEmpty()) {
return new Response(Response.BAD_REQUEST, "Key not found".getBytes(Charsets.UTF_8));
}
final ByteBuffer key = ByteBuffer.wrap(id.getBytes(Charsets.UTF_8));
logger.info("Request " + request.getMethod() + " with param: " + id);
try {
switch (request.getMethod()) {
case Request.METHOD_GET:
final ByteBuffer value = dao.get(key);
final ByteBuffer duplicate = value.duplicate();
byte [] body = new byte[duplicate.remaining()];
duplicate.get(body);
return new Response(Response.OK, body);
case Request.METHOD_PUT:
dao.upsert(key, ByteBuffer.wrap(request.getBody()));
return new Response(Response.CREATED, Response.EMPTY);
case Request.METHOD_DELETE:
dao.remove(key);
default:
return new Response(Response.METHOD_NOT_ALLOWED, Response.EMPTY);
}
}
catch (IOException e) {
return new Response(Response.INTERNAL_ERROR, Response.EMPTY);
}
catch (NoSuchElementException e) {
return new Response(Response.NOT_FOUND, "Key not found".getBytes(Charsets.UTF_8));
}
}
private static HttpServerConfig getConfig(final int port) {
if(port <= 1024 || port >= 65535) {
throw new IllegalArgumentException("Invalid port");
}
AcceptorConfig acceptorConfig = new AcceptorConfig();
acceptorConfig.port = port;
HttpServerConfig config = new HttpServerConfig();
config.acceptors = new AcceptorConfig[]{acceptorConfig};
return config;
}
} `
And server not running...
Do you have any plans to add quic and http3 support?
Hi, Andrew!
Trying to use one-nio 1.1.0 in my project, I faced with the problem: maven doesn't build the project because, since JDK9, tools.jar were removed from the distributive.
Do you have any workaround for that?
I have stock Debian 11 with kernel 5.10.162, compiled with glibc 2.31.
All versions of one-nio prior to 1.6.1 worked on my setup.
But since 1.6.1 it requires at least glibc 2.34 (ldd libonenio.so says "version `GLIBC_2.34' not found").
Please consider rebuild project with old version of glibc.
Who owns rights for source code?
I am looking at the json serialization. A bit surprised I am not finding an option to skip serialization of default values. Is this by design?
See HTTP/1.1:
Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non- identity transfer-coding, the Content-Length MUST be ignored.
Please make a tutorial and docs: how to create in-memory (off-heap) objects, how to allocate, how to serialize/deserialize (if need, i can't understand) this to byte array (located off-heap), how to change it fields.
For example now (in current system) I have class
User
{
long Id;
List[ItemA] ItemsA; // class ItemA { .. }
List[ItemB] ItemsB; // class ItemB { .. }
// and many more field
}
// and global collection
static HashMap[long, User] AllUsers;
// .. dynamically created users, dynamically filled ItemsA, B, ..
as I understand, I need use SharedMemoryMap.java or subclasses, but can't understend how.
and how to allocate new User, it's new ItemA, how to put, get to User.ItemsA collection.
is it enough to replace my static HashMap[long, User] AllUsers into SharedMemoryMap[long, User] or not?
should I see https://github.com/odnoklassniki/shared-memory-cache ? can't understand where is the full cache/allocate code.
When RpcSession fails with deserialization of incoming request, in particular, when some method or class is missing on server side, it throws ClassNotFoundException and tries to RpcSession#writeResponse with this exception. But RpcSession#writeResponse fails too, because ClassNotFoundException cannot be serialized.
It leads to the following exception on server side:
16:23:18,095|W|[NIO Selector #2] Local type not found: EchoService$Message
16:23:18,117|E|[NIO Selector #2] Cannot process session from 127.0.0.1
java.io.IOException: writeObject() is not fully supported. See implementation notes.
at one.nio.serial.gen.NullObjectOutputStream.unsupported(NullObjectOutputStream.java:159)
at one.nio.serial.gen.NullObjectOutputStream.putFields(NullObjectOutputStream.java:50)
at java.base/java.lang.ClassNotFoundException.writeObject(ClassNotFoundException.java:144)
at sun.reflect.Delegate0_ClassNotFoundException.calcSize(Unknown Source)
at one.nio.serial.GeneratedSerializer.calcSize(GeneratedSerializer.java:116)
at one.nio.serial.CalcSizeStream.writeObject(CalcSizeStream.java:129)
at one.nio.rpc.RpcSession.writeResponse(RpcSession.java:196)
at one.nio.rpc.RpcSession.handleDeserializationException(RpcSession.java:250)
at one.nio.rpc.RpcSession.processRead(RpcSession.java:108)
at one.nio.net.Session.process(Session.java:222)
at one.nio.server.SelectorThread.run(SelectorThread.java:70)
Looks like libonenio.so
is missing in jar deployed in Maven
This library looks fantastic. It would be great to see the slides (http://www.slideshare.net/AndreiPangin/pangin-highload) / video in English.
Hi!
After running my app, output to log:
мар 31, 2018 5:35:31 PM one.nio.serial.Repository addOptionalBootstrap
WARNING: Missing optional bootstrap class: one.app.remote.reflect.MethodId
мар 31, 2018 5:35:31 PM one.nio.serial.Repository addOptionalBootstrap
WARNING: Missing optional bootstrap class: one.app.remote.comp.RemoteMethodCallRequest
I see source code and found row on one.nio.serial.Repository:
addOptionalBootstrap("one.app.remote.reflect.MethodId");
addOptionalBootstrap("one.app.remote.comp.RemoteMethodCallRequest");
This rows is garbage?
Thanks.
Hi Andrey,
What do you think about moving of OffheapMap, SharedMemoryMap and etc. mem.* to separate repository to use them without one-nio?
Thanks.
The RpcClient can't acquire a socket at the first time after the peer restarted, because the IOException is thrown when the client invokes the remote method, and the RpcClient only catch SocketException instead of IOException in one.nio.rpc.RpcClient:
private byte[] invokeRaw(Object request, int timeout) throws Exception {
byte[] buffer = serialize(request);
Socket socket = borrowObject();
try {
try {
// Issue: stale connection(eg. the remote has been restored after the peer closed)
//and IOException occurs but can't be catched
sendRequest(socket, buffer, timeout);
} catch (SocketException e) {
// Stale connection? Retry on a fresh socket
destroyObject(socket);
socket = createObject();
sendRequest(socket, buffer, timeout);
}
But the sendRequest() method can throw IOException after the peer socket closed:
private void sendRequest(Socket socket, byte[] buffer, int timeout) throws IOException {
if (timeout != 0) socket.setTimeout(timeout);
socket.writeFully(buffer, 0, buffer.length);
socket.readFully(buffer, 0, 4);
}
This issue also exists in one.nio.http.HttpClient.invoke() method:
public Response invoke(Request request) throws InterruptedException, PoolException, IOException, HttpException {
int method = request.getMethod();
byte[] rawRequest = request.toBytes();
ResponseReader responseReader;
Socket socket = borrowObject();
boolean keepAlive = false;
try {
try {
// Issue: stale connection(eg. the remote has been restored after the peer closed)
//and IOException occurs but can't be catched
socket.writeFully(rawRequest, 0, rawRequest.length);
responseReader = new ResponseReader(socket, BUFFER_SIZE);
} catch (SocketException e) {
// Stale connection? Retry on a fresh socket
destroyObject(socket);
socket = createObject();
socket.writeFully(rawRequest, 0, rawRequest.length);
responseReader = new ResponseReader(socket, BUFFER_SIZE);
}
Good afternoon, comrades, tell me about the use of distributed cards.
I have two test app instances that declare a shared map across 1 and the same file.
When writing data from the first application to the SharedMemoryStringMap and receiving, I see the recorded data.
I'm launching a second application that tries to read data using a key that was written by the first application in SharedMemoryStringMap. As a result, I get a zero result.
The second application can see the data in the file only after the first application calls the map.close () method.
Tell me what can I do wrong?
Current use case:
JVM 1:
final String file = "./data.dat";
final SharedMemoryStringMap<String> map = new SharedMemoryStringMap<>(65535, file, 4 * M);
map.setSerializer(String.class);
map.put("12345", "test-shared-str");
String s = map.get("12345");
System.out.println(s);
// start daemon thread ...
JVM 2:
final String file = "./data.dat";
final SharedMemoryStringMap<String> map = new SharedMemoryStringMap<>(65535, file, 4 * M);
map.setSerializer(String.class);
String s = map.get("12345"); <-- null value
System.out.println(s);
I find the Socket is missing join/leave multicast support. Any plans to add it? Why was this left out? Would it be fine if this gets included?
I made a quick try without native support here 65bc2aa. Would the API changed be acceptable?
Just want to mention that in JDK 16, illegal-access=deny is the default, this means that getPollMethodHandle will return null.
It is possible to just use illegal-access=permit but would be nicer not to need to add a custom JVM parameter.
Example:
var byt = one.nio.serial.Serializer.persist(BigDecimal.ZERO);
Error:
Exception in thread "main" java.lang.VerifyError: Bad invokespecial instruction: current class isn't assignable to reference class.
Exception Details:
Location:
sun/reflect/Delegate0_BigDecimal.write(Ljava/lang/Object;Lone/nio/serial/DataStream;)V @9: invokespecial
Reason:
Error exists in the bytecode
Bytecode:
0000000: 2bc0 0015 4c2b b200 1bb7 001f 2c2b b400
0000010: 23b6 0031 2c2b b400 34b6 0038 b1
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3572)
at java.base/java.lang.Class.getConstructor0(Class.java:3777)
at java.base/java.lang.Class.getDeclaredConstructor(Class.java:2955)
at one.nio.serial.gen.DelegateGenerator.instantiate(DelegateGenerator.java:108)
at one.nio.serial.GeneratedSerializer.<init>(GeneratedSerializer.java:57)
at one.nio.serial.Repository.generateFor(Repository.java:379)
at one.nio.serial.Repository.get(Repository.java:182)
at one.nio.serial.PersistStream.writeObject(PersistStream.java:52)
at one.nio.serial.Serializer.persist(Serializer.java:140)
Version one-nio: 1.7.1
Looking at Utf8 code. I love the simplicity. But it looks like there is no support for 4 byte utf-8 characters. Came across a situation where 4 byte utf8 character was incorrectly encoded. Character in question https://www.compart.com/en/unicode/U+1F3A9.
Any chance to add 4 byte utf8 support in
https://github.com/odnoklassniki/one-nio/blob/ab2e0a6adbc3017540dc7e6691a11331fb1ed942/src/one/nio/util/Utf8.java?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.