gwhalin / memcached-java-client Goto Github PK
View Code? Open in Web Editor NEWInformation about this project lives on the wiki
Home Page: http://wiki.github.com/gwhalin/Memcached-Java-Client/
Information about this project lives on the wiki
Home Page: http://wiki.github.com/gwhalin/Memcached-Java-Client/
for your java docs, getMulti return a map, if key are not found, the key is not entered into the hashmap.
It's no problem at 2.5.1, but in 2.5.3 getMulti return null value in a map that key are not found.
eg. client.getMulti(new String[] {"xxx"}) , results are:
{} in 2.5.1,
{xxx=null} in 2.5.3
Test tools:jakarta-jmeter-2.3.2
Thread number: 100,500,1000
Error:
java.lang.NullPointerException
at com.schooner.MemCached.SchoonerSockIOPool.getConnection(SchoonerSockIOPool.java:590)
at com.schooner.MemCached.SchoonerSockIOPool.getSock(SchoonerSockIOPool.java:514)
at com.schooner.MemCached.AscIIClient.set(AscIIClient.java:376)
at com.schooner.MemCached.AscIIClient.set(AscIIClient.java:241)
at com.danga.MemCached.MemCachedClient.set(MemCachedClient.java:603)
at test.TestMemcachedSet.setkey(TestMemcachedSet.java:126)
at test.TestMemcachedSet.runTest(TestMemcachedSet.java:101)
at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:161)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:290)
at java.lang.Thread.run(Unknown Source)
TestCode:
public class TestMemcachedSet extends AbstractJavaSamplerClient {
private SampleResult results;
private String key;
private String value;
static SockIOPool pool;
static MemCachedClient mc;
private String serverlist;
private static int timeout;
public void setupTest(JavaSamplerContext arg0) {
results = new SampleResult();
timeout = arg0.getIntParameter("timeout");
key = arg0.getParameter("key");
value = arg0.getParameter("value");
serverlist = arg0.getParameter("serverlist");
if (key != null && key.length() > 0) {
results.setSamplerData(key);
}
if (value != null && value.length() > 0) {
results.setSamplerData(value);
}
if (serverlist != null && serverlist.length() > 0) {
results.setSamplerData(serverlist);
}
}
public Arguments getDefaultParameters() {
Arguments params = new Arguments();
params.addArgument("serverlist", "172.20.57.167:11211,172.20.57.112:11211");
params.addArgument("timeout", "5000");
params.addArgument("key", "key");
params.addArgument("value", "value");
return params;
}
public SampleResult runTest(JavaSamplerContext arg0) {
results.sampleStart();
boolean result=false;
try{
results.setRequestHeaders(key);
result = setkey(key,value);
results.setResponseMessage("set data:"+key+" is "+result);
results.setSuccessful(result);
}catch(Exception e){
e.printStackTrace();
results.setSuccessful(result);
}finally {
// Record end time and populate the results.
results.sampleEnd();
}
return results;
}
public void teardownTest(JavaSamplerContext arg0) {
//nothing todo
}
private boolean setkey(String key,String value)
{
pool = SockIOPool.getInstance();
pool.setServers(serverlist.split(","));
Integer[] weights={1,2};
pool.setWeights(weights);
pool.setNagle( false );
pool.setSocketTO(timeout );
pool.setHashingAlg(2);
pool.initialize();
mc = new MemCachedClient();
mc.setPrimitiveAsString(true);
mc.setSanitizeKeys(false);
boolean result = mc.set(key, value);
pool.shutDown();
return result ;
}
}
Memcached-Java-Client 2.6.3
In code com.schooner.MemCached.AscIIClient
line 580: String line = new SockInputStream(sock, Integer.MAX_VALUE).getLine().split("\r\n")[0];
line 585: } else if (NOTFOUND.equals(line)) {
And com.danga.MemCached.MemCachedClient.NOTFOUND = "NOT_FOUND\r\n", but variable line = "NOT_FOUND", so the result of line 585 is always FALSE.
I use "NOTFOUND.startsWith(line)" to instead of "NOTFOUND.equals(line)" temporarily.
In gets method of AscIIClient.java
case 1:
break;
case 2:
flag = Byte.parseByte(sb.toString());
break;
sb is parsed as bytes instead of Integer so throwing NumberFormatException
I'm able to use MemCachedClient.statsCacheDump successfully when connecting to a Memcached server using the ASCII protocol, but no items are returned when connecting using the binary protocol with SASL. Other operations work over the binary protocol with SASL, including the stats, statsSlabs and statsItems.
I'm using java-memcached-2.6.2.jar from the performance branch with JDK 6u26 32-bit on Linux. The Memcached server is 1.4.6 with libevent 2.0.12-stable. It is running with the following options:
memcached -d -p 11211 -u nobody -m 256 -c 1024 -P /var/run/memcached/memcached.pid -S
After upgrading to 2.6.2, we're intermittently getting the below exceptions (against 1.4.5 server).
ERROR 2011-11-28 23:00:01,141 [com.danga.MemCached.MemCachedClient 740206481@qtp-1696446139-18735] For input string: "command"
java.lang.NumberFormatException: For input string: "command"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:449)
at java.lang.Integer.parseInt(Integer.java:499)
Looked into the code and found that it should be due to a breakdown of the below typical 'get' conversation with the server. Eg. a HIT yields
set the_key 1 0 13
Just a number
get the_key
VALUE the_key 32 13
Just a number
END
For the exception scenario, it seems that the AsciiClient was trying to pass a response from the server that looked like "VALUE the_key command xxx". Wondering where did the 'command' keyword came from.
Anyone facing the same issue?
Could you please deploy Memcached-Java-Client-xxx.jar to maven central, in order that developers using maven get more convenience.
thx
I just noticed the latest update of the memcached-java-client after playing around with spymemcached and then reading about the performance increases. Great work!
I do have one issue...
With 2.0.1, I was able to set objects > 1 Mb by increasing the slab max (passing in the -I flag to memcached). With 2.5.1, the set command returns false if the object size is > 1 Mb. I checked the source code and didn't see a limit being set.
Any ideas? Thanks for you time.
Owen
I'm running memcached version 1.4.3 with the following settings
PORT="11211"
USER="nobody"
MAXCONN="1024"
OPTIONS="-I 8m"
CACHESIZE=1536
To test, I created a Junit test and assertTrue on the set command for byte arrays of size 1000000 and 2000000.
Hi, recently, when using memcached-client-java-3.01, I encountered some exceptions as follows:
Exception one:
31065671 [http-10.108.72.170-8080-exec-7] ERROR com.whalin.MemCached.MemCachedClient - ++++ exception thrown while trying to get object from cache for key: _22_channel_id=800000_query=|婴|乐|岛_sort=1_state=0_0
31065671 [http-10.108.72.170-8080-exec-7] ERROR com.whalin.MemCached.MemCachedClient - For input string: "command"
java.lang.NumberFormatException: For input string: "command"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:449)
at java.lang.Integer.parseInt(Integer.java:499)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.whalin.MemCached.MemCachedClient.get(Unknown Source)
Exception two:
22153145 [http-10.108.72.170-8080-exec-6] ERROR com.whalin.MemCached.MemCachedClient - Broken pipe
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69)
at sun.nio.ch.IOUtil.write(IOUtil.java:26)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.flush(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.whalin.MemCached.MemCachedClient.get(Unknown Source)
Exception three:
22152935 [ async update multicache] ERROR com.whalin.MemCached.MemCachedClient - ++++ exception thrown while writing bytes to server on set
22152935 [ async update multicache] ERROR com.whalin.MemCached.MemCachedClient - null
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:472)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:219)
at com.schooner.MemCached.SockInputStream.read(Unknown Source)
at com.schooner.MemCached.SockInputStream.getLine(Unknown Source)
at com.schooner.MemCached.AscIIClient.set(Unknown Source)
at com.schooner.MemCached.AscIIClient.set(Unknown Source)
at com.whalin.MemCached.MemCachedClient.set(Unknown Source)
When using version 2.5.3, all the best! Now don't know the reason, please help me!
I've just been running some tests on 2.5.0, to investigate the binary protocol, and I've hit an Out of Memory exception that occurs if a node failure is simulated, while the application is running, i.e.:
During the second lot of 500 get() you will recieve an OOM:
Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:633) at java.nio.DirectByteBuffer.(DirectByteBuffer.java:95) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288) at com.schooner.MemCached.SchoonerSockIO.(SchoonerSockIO.java:51) at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.(SchoonerSockIOPool.java:1369) at com.schooner.MemCached.SchoonerSockIOPool.createSocketWithAdd(SchoonerSockIOPool.java:404) at com.schooner.MemCached.SchoonerSockIOPool.getConnection(SchoonerSockIOPool.java:545) at com.schooner.MemCached.SchoonerSockIOPool.getSock(SchoonerSockIOPool.java:480) at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:1519) at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:750) at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:746) at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:1047)
Looking at the code, it seems to be coming from the ByteBuffer.allocateDirect in the SchoonerSockIO, constructor. The following patch in SchoonerSockIOPool.java (1375), seems to sort it out:
from
sock = getSocket(ip[0], Integer.parseInt(ip[1]), connectTimeout);
to
// get socket: default is to use non-blocking connect try { sock = getSocket(ip[0], Integer.parseInt(ip[1]), connectTimeout); } catch(Exception e) { writeBuf.clear(); writeBuf = null; throw new IOException(e); }
Seems to sort it out; but not sure if that causes any other issues.
Sorry if this post is messy, not sure of the markup to be used.
/dom
Can't find them here, don't see them in the download package. I can't find the site which used to host this library either.
I am working on the latest release java_memcached-release_2.5.3.jar.
I have added lots of cache in Memche,
and at least 50 get call per request.
recently, my site get very slow,
and get lots of error message:
com.schooner.MemCached.SchoonerSockIOPool Tue May 03 21:18:37 PDT 2011 - ++++ failed to get SockIO obj for:
it seems it try to connect the memcached server for every get request when server DOWN.
although I have config :
pool.setFailback(false);
so how can I stop it trying to connect dead server?
another issue:
protected final SchoonerSockIO createSocketWithAdd(String host) { SchoonerSockIO socket = null; try { poolCurrentConn.get(host).addAndGet(1); if (isTcp) { socket = new TCPSockIO(this, host, bufferSize, this.socketTO, this.socketConnectTO, this.nagle, true); } else { socket = new UDPSockIO(this, host, bufferSize, socketTO, true); } } catch (Exception ex) { log.error("++++ failed to get SockIO obj for: " + host); // log.error(ex.getMessage(), ex); socket = null; poolCurrentConn.get(host).decrementAndGet(); } return socket; }
will it clean up the direct memory if connection fail?
I have got some of this exception:
Caused by: java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:632) at java.nio.DirectByteBuffer.(DirectByteBuffer.java:95) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288) at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.(Unknown Source) at com.schooner.MemCached.SchoonerSockIOPool.createSocketWithAdd(Unknown Source) at com.schooner.MemCached.SchoonerSockIOPool.getConnection(Unknown Source) at com.schooner.MemCached.SchoonerSockIOPool.getSock(Unknown Source) at com.schooner.MemCached.AscIIClient.get(Unknown Source) at com.schooner.MemCached.AscIIClient.get(Unknown Source) at com.schooner.MemCached.AscIIClient.get(Unknown Source) at com.danga.MemCached.MemCachedClient.get(Unknown Source) at tws.webapp.cache.MemcachedCache.get(Unknown Source)
I can't debug Memcached-Java-Client with the source.jar,because I can't set breakpoint in my IDE,the code line in class and source is not the same?
I’ve been running the numbers to compare 2.0.1 and 2.5.0 and I’ve noticed something. This is against a simple single thread unit test with 64B, 1024B, 10240B, and 102400B. The first 3 perform as I would expect but at 102400b, it goes to crap. Not sure where the actual threshold is but above 10240B, each additional 10240B adds about an additional 2-3 seconds to the test.
With client 2.0.1
testCaching64B
testCaching1024B
testCaching10240B
testCaching102400B
Now with client 2.5.0 (same server):
testCaching64B
testCaching1024B
testCaching10240B
testCaching102400B
(This was on an 8 way, linux, 64bit box with 16gb ram)
Are others seeing this? Do you have any idea what might cause this? Based on these numbers, we won’t be able to upgrade.
Hi All,
I got a strange problem using java memcached client. I have a resin which has 512 resin-tcp-connection, all are runnable, but they are hanging at mc socket reading. So the resin is no response, for it reaches max 512 http request. And it just happens on busy resin, which means the request num is >30/s, and the network traffic is >100Mb/s. There are 5*50=250 MC connections.It seemed that readFromChannel sometimes not work. the possiblity is less than 1/100000.
But once it ocurr, the resin thread will keep on reading but read on data. So these fake dead threads will be more and more.
I also find someone else suffered similar problem about sun.nio.ch.FileDispatcher.read0, such as hadoop, jboss.
How to solve it? As the thread keep on reading but no timeout? Now I can only restart resin every day to avoid resin no responding.
And I was confuse about the code on com.schooner.MemCached.SchoonerSockIO.java
// the datagram sent from memcached mustn't beyond 1400 bytes.
public ByteBuffer readBuf = ByteBuffer.allocateDirect(8 * 1024);
comment: the MTU on linux is 1500 by default. why you say 1400 bytes?
if the datagram is about 1400 bytes, why sets the buff 8_1024, not 4_1024?
"resin-tcp-connection-10.67.222.61:8388-19351" daemon prio=10 tid=0x00002aabba08f800 nid=0x13c0 runnable [0x00002aabf8906000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.FileDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233)
at sun.nio.ch.IOUtil.read(IOUtil.java:200)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
- locked <0x00002aaadb561880> (a java.lang.Object)
at com.schooner.MemCached.SockInputStream.readFromChannel(SockInputStream.java:124)
at com.schooner.MemCached.SockInputStream.read(SockInputStream.java:109)
at com.schooner.MemCached.SockInputStream.getLine(SockInputStream.java:173)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:771)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:613)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:609)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:1047)
at cn.org.dbcache.BaseManager.getFromMemCachedServer(BaseManager.java:1806)
at cn.org.dbcache.BaseManager.findById(BaseManager.java:434)
at cn.org.dbcache.BaseManager.getFromMemCachedServerList(BaseManager.java:1883)
at cn.org.dbcache.BaseManager.loadList(BaseManager.java:1317)
gwhalin's memcached client implements a Logger itself.
But the logger is really weak. Could it support some mechanism which enables users to apply an alternative Logger implementation such as log4j?
In our production system, we have seen the following stack trace recently. We running the latest Memcached-Java-Client binary version 2.5.1.
2010-07-02 18:43:03 [TP-Processor61] ERROR c.s.j.s.c.ContainerResponse - The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633) ~[na:1.6.0_18]
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:95) ~[na:1.6.0_18]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288) ~[na:1.6.0_18]
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.SchoonerSockIOPool.createSocketWithAdd(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.SchoonerSockIOPool.getConnection(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.SchoonerSockIOPool.getSock(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.BinaryClient.get(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.BinaryClient.get(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.BinaryClient.get(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.danga.MemCached.MemCachedClient.get(Unknown Source) ~[memcached-client-2.5.1.jar:na]
2.6.6 stores a String value with flag=32 even with setPrimitiveAsString(true).
a python client (installed with pip install python-memcached) can not get the value.
MemCached: unknown flags on get: 20
13: handler exception: local variable 'val' referenced before assignment
i guess it's better to use flag=0 for setPrimitiveAsString(true) clients as it seems that
older versions used to do.
We wrongly removed some construction function of MemCachedClient, so there are some potential compatibility issue.
refer to the commit:
There are some parts of our code which specify a port and connect. This clashes with open mem-cached ports sometimes. We handle it the following way currently
SockIOPool.java
protected static Socket getSocket(String host, int port, int timeout) throws IOException {
SocketChannel sock = SocketChannel.open();
sock.socket().setReuseAddress(true);
sock.socket().bind(new InetSocketAddress(getNextPortFromRange()));
sock.socket().connect(new InetSocketAddress(host, port), timeout);
return sock.socket();
}
TCPSockIO.java
public final static Socket getSocket(String host, int port, int timeout) throws IOException {
SocketChannel sock = SocketChannel.open();
sock.socket().setReuseAddress(true);
sock.socket().bind(new InetSocketAddress(getNextPortFromRange()));
sock.socket().connect(new InetSocketAddress(host, port), timeout);
return sock.socket();
}
It would be great if we can expose an optional port range specifier as a config option to use.
Hi, Everyone:
I made a minor change on MemcachedPerfTest to test 'getMulti' instead of 'get', and then when I run the tests using thread number 50, runs number 100, I got the following exception sometimes:
Exception in thread "Thread-0" java.lang.NullPointerException at com.schooner.MemCached.AscIIClient$NIOLoader.doMulti(AscIIClient.java:1522) at com.schooner.MemCached.AscIIClient.getMulti(AscIIClient.java:1082) at com.schooner.MemCached.AscIIClient.getMulti(AscIIClient.java:1005) at com.danga.MemCached.MemCachedClient.getMulti(MemCachedClient.java:1207) at com.danga.MemCached.MemCachedClient.getMulti(MemCachedClient.java:1188) at com.schooner.MemCached.MemcachedPerfTest$bench.run(MemcachedPerfTest.java:123)
Then I did some researches and found there's a bug:
try { #1458 selector = Selector.open(); // get the sockets, flip them to non-blocking, and set up data // structures #1462 conns = new Connection[sockKeys.keySet().size()]; ... } catch (IOException e) { #1508 e.printStackTrace(); return; } finally { .... #1522 for (Connection c : conns) { if (c != null) c.close(); } } }
The exception occurred at #1522, it means conns is null, the possible reason is #1458 threw an exception and was not successfully run, so I added code #1508 for tracing the reason, and I got the stack of real exception:
java.io.IOException: Unable to establish loopback connection at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:106) at java.security.AccessController.doPrivileged(Native Method) at sun.nio.ch.PipeImpl.(PipeImpl.java:122) at sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:27) at java.nio.channels.Pipe.open(Pipe.java:133) at sun.nio.ch.WindowsSelectorImpl.(WindowsSelectorImpl.java:104) at sun.nio.ch.WindowsSelectorProvider.openSelector(WindowsSelectorProvider.java:26) at java.nio.channels.Selector.open(Selector.java:209) at com.schooner.MemCached.AscIIClient$NIOLoader.doMulti(AscIIClient.java:1458) at com.schooner.MemCached.AscIIClient.getMulti(AscIIClient.java:1082) at com.schooner.MemCached.AscIIClient.getMulti(AscIIClient.java:1005) at com.danga.MemCached.MemCachedClient.getMulti(MemCachedClient.java:1207) at com.danga.MemCached.MemCachedClient.getMulti(MemCachedClient.java:1188) at com.schooner.MemCached.MemcachedPerfTest$bench.run(MemcachedPerfTest.java:123) Caused by: java.net.BindException: Address already in use: connect at sun.nio.ch.Net.connect(Native Method) at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507) at java.nio.channels.SocketChannel.open(SocketChannel.java:146) at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:78) ... 13 more
Not sure why nio throws out this exception, probably the concurrent load is too high?(but I don't think 50 concurrent is very high ), anyway I think we should not let null pointer exception happen, it's a bug.
Please add a storeCounter with an expiry option.
In a 2006 message Greg wrote:
the storeCounter() method does not set an expiration, however, the
storeCounter() method is basically just a wrapper for set(), so an
expiration could most definitely be set.
http://lists.danga.com/pipermail/memcached/2006-March/002076.html
I am currently doing it manually and it would be nice to make this part of the API.
Memcached-Java-Client 1.6
java version "1.6.0_03"
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) Client VM (build 1.6.0_03-b05, mixed mode, sharing)
It's IdentityHashMap bug in the IdentityHashMapIterator.remove() method in jdk5, 6,
can also see https://issues.apache.org/jira/browse/DIRMINA-452, http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6612102
can change the use of Memcached-Java-Client?
call stack
"worker-15" daemon prio=10 tid=0x080f1800 nid=0x2d6b runnable [0xb483f000..0xb483ffb0]
java.lang.Thread.State: RUNNABLE
at java.util.IdentityHashMap.put(IdentityHashMap.java:421)
at com.danga.MemCached.SockIOPool.addSocketToPool(SockIOPool.java:961)
at com.danga.MemCached.SockIOPool.checkIn(SockIOPool.java:1045)
- locked <0x727fccc0> (a com.danga.MemCached.SockIOPool)
at com.danga.MemCached.SockIOPool.checkIn(SockIOPool.java:1060)
at com.danga.MemCached.SockIOPool$SockIO.close(SockIOPool.java:1520)
at com.danga.MemCached.MemCachedClient.set(MemCachedClient.java:780)
at com.danga.MemCached.MemCachedClient.set(MemCachedClient.java:473)
When I run the class MemCachedBench.java, I find sometimes the programme can't exit normally.
Then I debug it with eclipse, I found the "maintThread" is still waiting at line 1298, seems like dead lock.
protected void selfMaint() {
if ( log.isDebugEnabled() )
log.debug( "++++ Starting self maintenance...." );
// go through avail sockets and create sockets
// as needed to maintain pool settings
Map needSockets =
new HashMap();
************** synchronized( this ) {
// find out how many to create
for ( Iterator i = availPool.keySet().iterator(); i.hasNext(); ) {
I think maybe you shouldn't use "synchronized" in Method shutDown(), or you should use "this.wait(500);" to replace "Thread.sleep( 500 );".
It's easy to make mistakes with counter operations. The client should use primitive as String at all times with counters:
final MemCachedClient client = new MemCachedClient( true, true );
// fails without this line below
client.setPrimitiveAsString( true );
final String key = "testKey" + Math.random();
client.storeCounter( key, 3 );
client.incr( key );
// prints 4 if asString, -1 if not
System.out.println( client.getCounter( key ) );
When mysql memcached client set some data to memcached, other language memcached client can get the value,but Java memcached client whth version 2.5 and 2.6 occuer following error and return null (version 2.0.1 runs well):
com.danga.MemCached.MemCachedClient Tue Aug 16 11:17:56 CST 2011 - ++++ exception thrown while trying to get object from cache for key: test117
com.danga.MemCached.MemCachedClient Tue Aug 16 11:17:56 CST 2011 - invalid stream header: 76616C75
java.io.StreamCorruptedException: invalid stream header: 76616C75
at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
at java.io.ObjectInputStream.(Unknown Source)
at com.schooner.MemCached.ObjectTransCoder.decode(ObjectTransCoder.java:57)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:761)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:612)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:608)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:1047)
at TestM.main(TestM.java:56)
I found this problem caused by the code :
AscIIClient.java at line 755:
...
if(transCoder != null)
{
InputStream in = input;
if((flag & 2) == 2)
in = new GZIPInputStream(in);
if(classLoader == null)
o = transCoder.decode(in);
else
o = ((ObjectTransCoder)transCoder).decode(in, classLoader);
}
}
...
the code can't judge 'primitiveAsString',so the error happened.I modified the code like this, everything is normal:
...
else if (transCoder != null) {
// decode object with default transcoder.
InputStream in = input;
//
byte[] buf = ((SockInputStream) in).getBuffer();
if ((flag & F_COMPRESSED) == F_COMPRESSED)
in = new GZIPInputStream(in);
if (primitiveAsString || asString) {
o = new String(buf, defaultEncoding);
} else{
if (classLoader == null)
o = transCoder.decode(in);
else
o = ((ObjectTransCoder) transCoder).decode(in, classLoader);
...
and getMulti() and doMulti() method is the same reason.
This thread is a continuation of issue #6
Mengli as you said, gets will return null in case of non existing value.
I guess the reason for introducing gets and cas is to avoid concurrency problems during add/update, Now if suppose 2 machines wants to add an entry simultaneously without race condition it is not possible as gets will return null (initial value will be null as the key wont exist. So we are forced to use put).
So present gets and sets address concurrent update problem and not Add.
One solution I could think of is to store the key and generate a cas value, if the key is not present, in memcache server itself
This "continue" need to be deleted?
// only stop when we see
// \r (13) followed by \n (10)
if (b[0] == 13) {
eol = true;
continue;
}
if (eol) {
if (b[0] == 10)
break;
eol = false;
}
Hi,
We are using the memcached client as part of our client which maintains dedicated connection via memcached client pool to memcached servers.
We observed that when the memcached servers are recycled, the connection pool on the client side becomes stale. The client will understand this only when its ready to write an entry to the recycled memcached server(s). The process of re-establishing/refreshing the client pool takes around 2 failed requests. The third consecutive request goes through.
First Request:
The first request fails with the following exception which is caught but swallowed in the catch(RuntimeException e) block in BinaryClient::set(byte opcode, String key, Object value, Date expiry, Integer hashCode, long casUnique, boolean asString) method.
Exception:
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:474)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:208)
at com.schooner.MemCached.SockInputStream.read(SockInputStream.java:110)
at java.io.DataInputStream.readInt(DataInputStream.java:370)
at com.schooner.MemCached.BinaryClient.set(BinaryClient.java:420)
at com.schooner.MemCached.BinaryClient.add(BinaryClient.java:249)
at com.danga.MemCached.MemCachedClient.add(MemCachedClient.java:568)
Second Request:
The second request fails with exception
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:72)
at sun.nio.ch.IOUtil.write(IOUtil.java:28)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.flush(SchoonerSockIOPool.java:3077)
at com.schooner.MemCached.BinaryClient.set(BinaryClient.java:417)
at com.schooner.MemCached.BinaryClient.add(BinaryClient.java:249)
at com.danga.MemCached.MemCachedClient.add(MemCachedClient.java:568)
We tried to set the errorhandler but were not able to receive the callback.
Third Request: Success.
The current implementation leaves at least two failed requests per memcached server, when the sevrers are recycled.
Is it possible to:
Thank you for your help.
Regards.
Bhargav.
Hi Mengli,
We are currently using the Memcached client and providing the pool parameters through our custom configuration system. We are using the client object as a spring singleton wired into our main codebase.
The issue we are facing is, we want to refresh the pool when any config value is changed in our config system.
In order to do so there should be a method in SchoonerSockIOPool class which will remove the pool object from "pools" HashMap. This method will take in poolname as argument and will remove that object from the HashMap.
If we can do this then we can remove the pool object if any config value is changed and then reinitialize the pool at runtime with new values. With addition of this feature we don't have to recycle our servers everytime a config value is changed.
I was wondering how difficult it would be for you to add this feature?
Thank you for your time.
Regards,
bhargav
Hi,
We are trying to use your memcached-java client to connect to the memcacheD server and we want to test following failover scenario.
Then when we reconnect the cable, it successfully resumes. So my question was, are we missing any configuration values?
btw I am setting the "setFailback" and "setFailover" flags to true.
here is my code snippet:
SchoonerSockIOPool pool = null;
try {
pool = SchoonerSockIOPool.getInstance("default", AuthInfo.typical("cacheuser", "password"));
pool.setServers( servers );
pool.setWeights( weights );
pool.setInitConn( 10 );
pool.setMinConn( 5 );
pool.setMaxConn( 250 );
pool.setMaintSleep( 10 );
pool.setNagle( false );
pool.setSocketTO( readTO );
pool.setSocketConnectTO( 0 );
pool.setMaxIdle( maxIdle );
pool.setFailback(true);
pool.setFailover(true);
pool.initialize();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
So is this client behavior expected?
We will highly appreciate your help on this issue.
Thank you for your time.
When SchoonerSockIOPool.getSoket() returns null we return false.
In case of normal commands (set replace append ...) this is of no problem but for commands like add and cas we cant differentiate cas and add failure with connection failure.
Instead we could throw an RuntimeException which would be a configurable data
Exception in thread "http-bio-8080-exec-14" java.lang.OutOfMemoryError
Java HotSpot(TM) Server VM warning: Attempt to allocate stack guard pages failed.
at sun.misc.Unsafe.allocateMemory(Native Method)
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:127)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.(Unknown Source)
at com.schooner.MemCached.SchoonerSockIOFactory.createSocket(Unknown Source)
at com.schooner.MemCached.SchoonerSockIOFactory.makeObject(Unknown Source)
运行环境 java_memcached-release_2.5.2, tomcat7, jdk7
配置参数-server -Xms1548m -Xmx1548m -Xmn1000m -XX:PermSize=400m -XX:MaxPermSize=400m -XX:ReservedCodeCacheSize=100m
操作系统 Linux version 2.6.18-164.el5 ([email protected]) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Tue Aug 18 15:51:48 EDT 2009
cpu 单核 内存 4g,使用2g左右
This is not really an issue but a question instead. It's said in memcached protocol that "cas unique" is a 64 bit integer. I have two questions regarding this:
Thanks.
Hi all,
Has anybody experienced problems using java client's getMultiArray()
method(Schooner memcached client 2.5.0)?
We run into this problem recently - getMultiArray() does not return
all objects specified. But when trying to load the missing objects
using get() method one by one, they are all fine. Those objects also
exist in memcached server with proper expiration time when checking
with 'stats cachedump'.
Any help appreciated, Thanks!
This is happening in the performance branch.
If I read the code correctly, a one time socket will be created if the pool is full and all the sockets in it are being used. Then in AsciiClient class, sock.close() is called in the finally clause of all the command methods. Since sock.close() will not actually close the socket and will only put pooled sockets back in the pool, these one time sockets are never closed. This is reproducible by making the number of concurrent clients much more than the pool size. I was getting "Too many open files" within two minutes. The problem disappeared after I applied a fix.
java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:501)
at java.nio.DirectByteBuffer.putLong(DirectByteBuffer.java:738)
at com.schooner.MemCached.BinaryClient$NIOLoader$Connection.(Unknown Source)
at com.schooner.MemCached.BinaryClient$NIOLoader.doMulti(Unknown Source)
at com.schooner.MemCached.BinaryClient$NIOLoader.doMulti(Unknown Source)
at com.schooner.MemCached.BinaryClient.getMulti(Unknown Source)
at com.danga.MemCached.MemCachedClient.getMulti(Unknown Source)
at com.danga.MemCached.MemCachedClient.getMulti(Unknown Source)
should calculate the size of keys, instead of hard code 10K bytes
Hi,
I am trying to use the setAliveCheck(true) flag so that when a memcacheD server goes down, the flag makes the client check for the connection and if not available then failover to the next server. During my execution I noticed that it hangs the application and the appln does nothing. I am trying to use the client with SASL support.
Thanks.
here is my code snippet:
SchoonerSockIOPool pool = null;
try {
pool = SchoonerSockIOPool.getInstance("default", AuthInfo.typical("cacheuser", "password"));
pool.setServers( servers );
pool.setWeights( weights );
pool.setInitConn( 10 );
pool.setMinConn( 5 );
pool.setMaxConn( 250 );
pool.setMaintSleep( 10 );
pool.setNagle( false );
pool.setSocketTO( readTO );
pool.setSocketConnectTO( 0 );
pool.setMaxIdle( maxIdle );
// pool.setAliveCheck(true);
pool.setFailback(true);
pool.setFailover(true);
pool.initialize();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
Thanks.
When creating a MemCachedClient against a bad/nonexistent memcached server, every call to the client will result in a file descriptor being claimed for a socket, but then never closed. These open FDs will stick around for the length of the process, even if the memcached server becomes valid.
Using 3.0.1, with tcp connection.
Looks to be caused by the following:
When requesting a connection from the pool, it will attempt to create a new TCPSockIO object (via GenericObjectPool.borrowObject()). Instantiation of TCPSockIO attempts to create and connect to a socket for the memcached server. But it does this as a two-step process: first opening the socket, then attempting to connect to it. If the connection throws an exception the socket is never closed. And since the TCPSockIO object was never fully instantiated, the object pool simply registers the exception and continues - with the next request repeating the process.
Fix should be relatively simple. Either use the SocketChannel convenience open method (which does not accept a timeout):
Or wrap the socket connect() call in a try/catch, and close if needed:
diff --git a/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java b/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java index 396894a..7f4abe0 100644 --- a/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java +++ b/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java @@ -1474,7 +1474,14 @@ public class SchoonerSockIOPool { */ protected final static Socket getSocket(String host, int port, int timeout) throws IOException { SocketChannel sock = SocketChannel.open(); + try { sock.socket().connect(new InetSocketAddress(host, port), timeout); + } catch (IOException e) { + sock.socket().close(); + throw e; + } return sock.socket(); }
For performance branch:
In master the storeCounter overloads set() with primitiveAsString as true. The performance branch currently doesn't do this so storeCounter will cause the server to throw the following error message unless the flag is set for the entire instance of the client:
CLIENT_ERROR cannot increment or decrement non-numeric value
Recently our developer was evaluating Memcached as an alternative solution on site specific problems. When we started stress test on Memcached with compress enabled, something went wrong.
We call the set(...) method of MemcachedClient within a loop with a large base, like one million times per batch; and we limited the Java heap size up to 2gb, the stack size up to 1024k. After the test began, the Resident Memory of this Java process inflated dramaticly in liner pattern, without the acknowledgement of heap limit. Our test server has 16GB physical memory, but in a short while, within 40 seconds approximately, this test Java process had taken up all the physical memory and throws OOM with the exception shows the zlib was just trying to allocate memory.
Then we've profiled the test process but found nothing abnormal, so we dumped the heap, and it was okay, only occupies 123mb, and the stack was also okay; as the GZIPOurputStream utilizing JNI call, we believe this problem was caused by over-inflation of native heap, the memory malloced by JNI function.
So we examined the GZIPOutputStream and found it extends DeflaterOutputStream and it use Deflater to compress stream, according to Java-doc of Deflater, the caller should invoke the end() method of Deflater object when the compressor is no longer being used or it will also be called automatically by the finalize() method. So here is the problem, now the MemcachedClient code does not explicitly invoke the end() method(which is indirectly invoked by the close() method of GZIPOutputStream), then the Java VM may not realize the mass memory occupied by JNI portion, and the Deflator object may not be GCed at first priority(in previously dumped heap, the Deflator class has approximately 6 thousand instances), at this point, the native heap memory was out of control.
To sum up, we insert a simple line after line 800:
gos.finish();
gos.close();
then the memory problem definitely gone, all seems good afterward, the Java process Resident Memory had never gone too far, just around 800mb.
So should we add this line to the trunk code?
I just noticed the latest update of the memcached-java-client after playing around with spymemcached and then reading about the performance increases. Great work!
I do have one issue...
With 2.0.1, I was able to set objects > 1 Mb by increasing the slab max (passing in the -I flag to memcached). With 2.5.1, the set command returns false if the object size is > 1 Mb. I checked the source code and didn't see a limit being set.
Any ideas? Thanks for you time.
Owen
Hi,when I try to get data from tokyo tyrant by java memcached,it allways return null,but correct when by spymemcached,so should I implement my own decoder for tokyo tyrant ? or is there any exsited solution for java memcached ? thank you very much~~
We use memcached-java-client as one of our memcached client in our project.we found that when we operation big data some thing goes wrong:
[2012-03-05 22:58:29.872]-[ERROR] com.danga.MemCached.MemCachedClient ++++ exception thrown while writing bytes to server on get
[2012-03-05 22:58:29.874]-[ERROR] com.danga.MemCached.MemCachedClient An existing connection was forcibly closed by the remote host
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:33)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69)
at sun.nio.ch.IOUtil.write(IOUtil.java:26)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.flush(SchoonerSockIOPool.java:1547)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:1482)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:738)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:734)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:917)
at com.eshore.crm.memcache.client.SimpleTest$1.run(SimpleTest.java:91)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
null
Test code:
memcached server:
/usr/local/bin/memcached -d -m 4096m -u mem -vv -p 11212 -I 10m
client test:
import java.util.UUID;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import com.danga.MemCached.MemCachedClient;
import com.danga.MemCached.SockIOPool;
public class SimpleTest {
// create a static client as most installs only need
// a single instance
protected static MemCachedClient mcc;
// set up connection pool once at class load
static {
// server list and weights
String[] servers = { "10.17.13.49:11212" };
Integer[] weights = { 3, 3, 2 };
// grab an instance of our connection pool
SockIOPool pool = SockIOPool.getInstance();
// set the servers and the weights
pool.setServers(servers);
pool.setWeights(weights);
// set some basic pool settings
// 5 initial, 5 min, and 250 max conns
// and set the max idle time for a conn
// to 6 hours
pool.setInitConn(5);
pool.setMinConn(5);
pool.setMaxConn(250);
pool.setMaxIdle(1000 * 60 * 60 * 6);
// set the sleep for the maint thread
// it will wake up every x seconds and
// maintain the pool size
pool.setMaintSleep(30);
// set some TCP settings
// disable nagle
// set the read timeout to 3 secs
// and don’t set a connect timeout
pool.setNagle(false);
pool.setSocketTO(3000);
pool.setSocketConnectTO(0);
pool.setBufferSize(30000000);
// initialize the connection pool
pool.initialize();
// lets set some compression on for the client
// compress anything larger than 64k
//mcc.setCompressEnable(true);
//mcc.setCompressThreshold(64 * 1024);
mcc = new MemCachedClient(true);
}
public static void main(String args[]){
int k=1;
String[] values= new String[8000];
int j = 0;
for (int i = 0; i < keys.length; i++) {
keys[j] = "3";
j++;
}
BlockingQueue<Runnable> startStopQueue = new LinkedBlockingQueue<Runnable>();
ThreadPoolExecutor taskExecutor = new ThreadPoolExecutor(10,
10, 10, TimeUnit.SECONDS,
startStopQueue);
taskExecutor.allowCoreThreadTimeOut(true);
final String[] vs=values;
final MemCachedClient mc = mcc;
while(k>0){
//mcc.delete("foo");
taskExecutor.submit(new Runnable(){
@Override
public void run() {
//mcc = new MemCachedClient(true);
String key = "foo"+UUID.randomUUID();
mcc.set(key, vs);
Object obj = mcc.get(key);
System.out.println(obj);
}
});
try {
TimeUnit.MICROSECONDS.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
k--;
}
taskExecutor.shutdown();
while(!taskExecutor.isTerminated()){
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.exit(1);
}
when I change the values size to 1000 the error won't happend.did something wrong with my code or config .please help me out !!
Hi, Everyone:
I took a look at the source code of Memcached-Java-Client 2.6.1 last night, and found it catches Exception in many places. I think which would probably cause some bugs when the thread is interrupted. a typical scene is:
The client application needs a timeout for calling each memcached get, so it will call the memcached 'get' in a separated child thread and wait it completed in main thread, if it's timed out, then the main thread needs to ignore the cache data(treat it as not hit cache) and terminate the child thread.
The current implementation in Memcached-Java-Client always just catches Exception but doesn't throw out interrupted exception again or restore the interrupted status of the thread when the caught exception is an interrupted exception. which would cause the child thread can not be terminated correctly and would cause system slow when there're lots of slow 'get' need to be terminated.
We are using Memcached-Java-Client 2.5.1 in a very big project which has more than 200 concurrent 'get' and 'set' very often, with more than 500K data for most of the access, there're performance issues for more than half a year. we have made some small changes on source code to fix some small bugs, but can not resolve the performance issues, and we are planning to use other clients recently. last night I was excited to see 2.6.1 was released a few days ago, hopefully it would resolve the performance issues we have.
It appears that if the client is configured to use SASL but the Memcached server does not require it, the connection succeeds. Is there any way to tell that the server does not require SASL, other than to try connecting without it first? Even a way to query the pool to see if the SASL credentials were actually used to make the connection would be useful.
Once the configuration setPrimitiveAsString is set true, the following code:
byte[] bytes = new byte[];
client.set(bytes);
will actually act as:
client.set(bytes.toString().getBytes[]); //according to line 392, AscIIClient.java
while "bytes.toString()" will return something like "[B@1837697", which is obviously wrong.
Is this a bug, or I'm using MemcacheClient in the wrong way?
Hi, Everyone:
I always get warnings when I call getMulti, the message is as follows:
WARN com.danga.MemCached.MemCachedClient - Caught the aforementioned exception on Connection to localhost:11211 with 1 bufs; done is true
Is it normal or any bug caused this?
I use ':' in the key, when I set / get, it doesn't work well.
Any body meet the same problem?
我通过Maven依赖Memcached,发现与提供的java_memcached-release_2.6.6版本中的包名不一致,但是里面的接口好像一致。
com.whalin.MemCached
com.danga.MemCached
这给我带来了许多困扰,让我不知道该如何选择,总之我认为这不是一个好的现象。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.