rtbkit / rtbkit Goto Github PK
View Code? Open in Web Editor NEWRTBkit is an open-source software package that allows you to create and deploy a Real Time Bidder for display advertising.
Home Page: http://rtbkit.org
License: Apache License 2.0
RTBkit is an open-source software package that allows you to create and deploy a Real Time Bidder for display advertising.
Home Page: http://rtbkit.org
License: Apache License 2.0
Hi! Great work with the project. I'm interested to test the system and also build my own on aws. For that, I'm looking for real time data stream. Is it possible for a place to fetch such stream to test our own system?
Will signing up with google adx or appnexus help?
Can the build be made on Amazon Linux/RHEL?
Hi, What is the purpose of sending the error reason to a stats server?
I'm talking about this line:
rtbkit/rtbkit/core/router/router.cc
Line 1279 in b67f5c4
In my side that was braking our server when being hit with reasons like the following:
[bid price 500USD/1M is lower then thebid floor prices 938USD/1M, or are in different currencies USD and USD 0 1446638717 ]
Hi,
I am getting the following while making the demo stack. Kindly help:
ailed to open file sample.launch.json
token.type_ = 0
cannot read file 'sample.launch.json'
Regards
Murtaza
Hi,
After coming across this on https://github.com/rtbkit/rtbkit/wiki/RTBkit-Deps-and-Ubuntu-14.04 one of the issues mentioned I don't need to build platform-deps and go straight to rtb-kit deps:
git clone https://github.com/rtbkit/rtbkit-deps.git
$ cd rtbkit-deps
$ git submodule update --init
Is that true? We don't need to use platform-deps anymore?
Hi,
We would like augment a bid request with some user data. I have gone through the frequency cap example but it is not very clear how to use that approach to augment user data. Say for example adding the gender value (M/F). Do you have any examples or samples on that?
I am particularly confused by this line in the frequency cap example.
result[account[0]].data = count;
How would the router know this field 'data' is used for frequency counting unless it is hardcoded?
Would it be okay if we just match the gender value we will get from the config object with the one from the user store and if it matches then insert "pass-frequency-cap-ex" tag? It is not clean and I would prefer a cleaner approach.
Any suggestions?
Thanks.
If for some reason a bad response is returned from downstream, looks like respDesc.parseJson() call on http_bidder_interface.cc:190 just bails out resulting in termination of the entire process. Just because the bidder is flaky should this really terminate the router, rather then perhaps removing the config(?) and logging error, allowing to redeploy the bidder interface without restarting others.
when i run:
sudo /opt/graphite/bin/carbon-cache.py start
I got the Warning:
/usr/lib/python2.7/dist-packages/zope/init.py:3: UserWarning: Module twisted was already imported from /home/rtbkit/.local/lib/python2.7/site-packages/twisted/init.pyc, but /opt/graphite/lib is being added to sys.path
import pkg_resources
Starting carbon-cache (instance a)
Also i find some discussion about this.
https://groups.google.com/a/rtbkit.org/forum/#!topic/discuss/eRZ2gjlMb8I
But it doesn't work for me.
Meanwhile, I can not find the code 'from twisted.scripts._twistd_unix import daemonize' in file '/opt/graphite/lib/carbon/util.py'.
So weird
anyone help?
Any suggestions pls ?
Both options have the value -1. This way it's impossible to determine if the delay was not provided, or it's generic mid roll.
Hello !
I'm trying to compile the rtbkit on a 32 bits architecture and failing to do so.
I got an invalid instruction suffix for "inc" error caused by line 119 of the atomic_ops.h file, called when reading data_aliases.cc
FYI I'm working on Ubuntu 14.04.
Do you have any idea of where this problem might come from ?
Thank you,
Ralph.
Hi,
The link on bidding agent regarding Banker's documentation is not working.
"
A more lengthy description of accounts can be found in the banker's documentation.
"
After running the demo stack and getting win bids from mock exchange. i have seen that Ad Server Shows "Broken Pipe" in logs.
What is the fix for it?
Regards
Murtaza
Hi, rtbkit made compiled successfully but while running make test it throws following exceptions:
[TESTCASE] redis_async_test
redis_async_test FAILED
Running 2 test cases...
starting redis temporary server under unique path build/x86_64/tmp/redis-temporary-server-30620-1
running redis
----------------- Exception thrown ------------------------
type: ML::Exception
pid: 30623; tid: 30623
what: redis failed to start: No such file or directory
stack:
00: 0x0x2b8435d7fde7 at __cxa_throw + 0x27 in build/x86_64/bin/libexception_hook.so + 0xde7build/x86_64/bin/libexception_hook.so(__cxa_throw+0x27) [0x2b8435d7fde7]
01: 0x0x4197f8 at Redis::RedisTemporaryServer::start() + 0x688 in build/x86_64/tests/redis_async_test + 0x197f8build/x86_64/tests/redis_async_test(_ZN5Redis20RedisTemporaryServer5startEv+0x688) [0x4197f8]
02: 0x0x41053e in build/x86_64/tests/redis_async_test + 0x1053ebuild/x86_64/tests/redis_async_test() [0x41053e]
03: 0x0x413330 at test_redis_async::test_method() + 0x50 in build/x86_64/tests/redis_async_test + 0x13330build/x86_64/tests/redis_async_test(_ZN16test_redis_async11test_methodEv+0x50) [0x413330]
04: 0x0x416401 in build/x86_64/tests/redis_async_test + 0x16401build/x86_64/tests/redis_async_test() [0x416401]
05: 0x0x416887 at boost::unit_test::ut_detail::callback0_impl_t<boost::unit_test::ut_detail::unused, void ()()>::invoke() + 0x7 in build/x86_64/tests/redis_async_test + 0x16887build/x86_64/tests/redis_async_test(_ZN5boost9unit_test9ut_detail16callback0_impl_tINS1_6unusedEPFvvEE6invokeEv+0x7) [0x416887]
06: 0x0x2b8436415131 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x65131/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(+0x65131) [0x2b8436415131]
07: 0x0x2b84363f95fe at boost::execution_monitor::catch_signals(boost::unit_test::callback0 const&) + 0xde in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x495fe/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost17execution_monitor13catch_signalsERKNS_9unit_test9callback0IiEE+0xde) [0x2b84363f95fe]
08: 0x0x2b84363f9efb at boost::execution_monitor::execute(boost::unit_test::callback0 const&) + 0x2b in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x49efb/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost17execution_monitor7executeERKNS_9unit_test9callback0IiEE+0x2b) [0x2b84363f9efb]
09: 0x0x2b843641522b at boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::unit_test::test_case const&) + 0xab in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x6522b/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost9unit_test19unit_test_monitor_t21execute_and_translateERKNS0_9test_caseE+0xab) [0x2b843641522b]
10: 0x0x2b84364009a1 at boost::unit_test::framework_impl::visit(boost::unit_test::test_case const&) + 0xb1 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x509a1/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost9unit_test14framework_impl5visitERKNS0_9test_caseE+0xb1) [0x2b84364009a1]
11: 0x0x2b8436431093 at boost::unit_test::traverse_test_tree(boost::unit_test::test_suite const&, boost::unit_test::test_tree_visitor&) + 0x53 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x81093/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(ZN5boost9unit_test18traverse_test_treeERKNS0_10test_suiteERNS0_17test_tree_visitorE+0x53) [0x2b8436431093]
12: 0x0x2b84363fc0d1 at boost::unit_test::framework::run(unsigned long, bool) + 0x551 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x4c0d1/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(ZN5boost9unit_test9framework3runEmb+0x551) [0x2b84363fc0d1]
13: 0x0x2b843641466f at boost::unit_test::unit_test_main(bool ()(), int, char) + 0xcf in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x6466f/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost9unit_test14unit_test_mainEPFbvEiPPc+0xcf) [0x2b843641466f]
14: 0x0x2b8436e8676d at __libc_start_main + 0xed in /lib/x86_64-linux-gnu/libc.so.6 + 0x2176d/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x2b8436e8676d]
15: 0x0x40e3d5 in build/x86_64/tests/redis_async_test + 0xe3d5build/x86_64/tests/redis_async_test() [0x40e3d5]
----------------- Exception thrown ------------------------
type: boost::execution_exception
pid: 30623; tid: 30623
stack:
00: 0x0x2b8435d7fde7 at __cxa_throw + 0x27 in build/x86_64/bin/libexception_hook.so + 0xde7build/x86_64/bin/libexception_hook.so(__cxa_throw+0x27) [0x2b8435d7fde7]
01: 0x0x2b84363f98ec in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x498ec/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(+0x498ec) [0x2b84363f98ec]
02: 0x0x2b84363f0772 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x40772/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(+0x40772) [0x2b84363f0772]
03: 0x0x2b84363fa001 at boost::execution_monitor::execute(boost::unit_test::callback0 const&) + 0x131 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x4a001/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost17execution_monitor7executeERKNS_9unit_test9callback0IiEE+0x131) [0x2b84363fa001]
04: 0x0x2b843641522b at boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::unit_test::test_case const&) + 0xab in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x6522b/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost9unit_test19unit_test_monitor_t21execute_and_translateERKNS0_9test_caseE+0xab) [0x2b843641522b]
05: 0x0x2b84364009a1 at boost::unit_test::framework_impl::visit(boost::unit_test::test_case const&) + 0xb1 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x509a1/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost9unit_test14framework_impl5visitERKNS0_9test_caseE+0xb1) [0x2b84364009a1]
06: 0x0x2b8436431093 at boost::unit_test::traverse_test_tree(boost::unit_test::test_suite const&, boost::unit_test::test_tree_visitor&) + 0x53 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x81093/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(ZN5boost9unit_test18traverse_test_treeERKNS0_10test_suiteERNS0_17test_tree_visitorE+0x53) [0x2b8436431093]
07: 0x0x2b84363fc0d1 at boost::unit_test::framework::run(unsigned long, bool) + 0x551 in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x4c0d1/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(ZN5boost9unit_test9framework3runEmb+0x551) [0x2b84363fc0d1]
08: 0x0x2b843641466f at boost::unit_test::unit_test_main(bool ()(), int, char*) + 0xcf in /home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0 + 0x6466f/home/ubuntu/local/lib/libboost_unit_test_framework.so.1.47.0(_ZN5boost9unit_test14unit_test_mainEPFbvEiPPc+0xcf) [0x2b843641466f]
09: 0x0x2b8436e8676d at __libc_start_main + 0xed in /lib/x86_64-linux-gnu/libc.so.6 + 0x2176d/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x2b8436e8676d]
10: 0x0x40e3d5 in build/x86_64/tests/redis_async_test + 0xe3d5build/x86_64/tests/redis_async_test() [0x40e3d5]
unknown location(0): fatal error in "test_redis_async": std::exception: redis failed to start: No such file or directory
starting redis temporary server under unique path build/x86_64/tmp/redis-temporary-server-30623-2
running redis
ubunt 14.02 LTS
gcc 4.8.2;
boost: 1.57.0 , I download from the http://www.boost.org/ , replace the boost-svn , and install it ;
when i run make compile NODEJS_ENABLED=0, there is always a error about link of boost like this:
" make compile NODEJS_ENABLED=0
[BIN] classifier_training_tool
/home/vincent/rtbkit/build/x86_64/bin/libutils.so: undefined reference to `boost::iostreams::file_descriptor::seek(long long, std::_Ios_Seekdir)'
collect2: error: ld returned 1 exit status
make: *** [build/x86_64/bin/classifier_training_tool] Error 1
"
I check the boost library on $home/local/, i think it is ok . Dose anyone meet this error ? Wish your help ,Thanks ~~
Hi,
I'm able to get the demo stack running, but I'm getting very few wins to appear. After leaving the stack running for about 5 minutes, the only wins I see are
$ ./build/x86_64/bin/mock_exchange_runner
publishing on port 12339
publishing on port 12340
win sent payload={"account":["hello","world"],"adSpotId":"1","auctionId":"35437","bidTimestamp":0.0,"channels":null,"timestamp":1370906736.140897,"type":1,"uids":{"prov":"bar","xchg":"foo"},"winPrice":[9,"USD/1M"]}
win sent payload={"account":[],"adSpotId":"1","auctionId":"35437","bidTimestamp":0.0,"channels":null,"label":"IMPRESSION","timestamp":1370906736.141844,"type":3,"uids":{"prov":"bar","xchg":"foo"},"winPrice":0}
Plus, the second win is to an empty account with no win price. The post-auction window (#8 in the tmux console) just displays
checking 0 submitted auctions for inferred loss
checking 4 finished auctions for expiry
checking 0 submitted auctions for inferred loss
checking 4 finished auctions for expiry
checking 0 submitted auctions for inferred loss
checking 4 finished auctions for expiry
and so on. No errors on compilation, except that the rtbkit_integration_test runs forever so I just run it manually (with "make rtbkit_integration_test" instead of "make test").
Are these results abnormal for the demo stack, and, if they are, what might be causing them?
hi
just like the title,
can i use perl, or python , or other programe language to devolop the agent , the exchange connector or the other modules ? and how to do this?
thanks!
Hi,
I am trying to setup rtbkit using the guide here:https://github.com/rtbkit/rtbkit/wiki/Demo-Stack.
Everything till the step to start launched seems to be working fine, but when I try to start the launcher using the command:
"./build/x86_64/bin/launcher --node localhost --script ./launch.sh --launch rtbkit/sample.launch.json"
I get the following errors:
2014-04-06 02:39:48.834 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch ad-server
2014-04-06 02:39:48.835 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch monitor
2014-04-06 02:39:48.837 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch logger
2014-04-06 02:39:48.839 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch agent-configuration
2014-04-06 02:39:48.841 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch banker
2014-04-06 02:39:48.843 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch augmentor
2014-04-06 02:39:48.845 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch router
2014-04-06 02:39:48.847 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch post-auction
2014-04-06 02:39:48.851 Launch ./soa/launcher/launcher.h:209 - void Datacratic::Launcher::Task::spawn(const string&) launch fixed-price-agent
2014-04-06 02:39:49.098 LaunchError ./soa/launcher/launcher.h:526 - static void Datacratic::Launcher::Service::sigchld(int) process 13480 killed by signal 6
2014-04-06 02:39:49.099 LaunchError ./soa/launcher/launcher.h:493 - void Datacratic::Launcher::Service::onDeath(int) crash! ad-server detected at Sun Apr 6 02:39:49 2014
2014-04-06 02:39:49.368 LaunchError ./soa/launcher/launcher.h:526 - static void Datacratic::Launcher::Service::sigchld(int) process 13494 killed by signal 6
Checking for logs in adserver, I get the following message:
terminate called after throwing an instance of 'ML::Exception'
what(): Zookeeper error on zoo_create, path /rtb-test/serviceClass/adServer/adServer: node exists
I tried to kill and restart zookeeper, but the issue didn't get resolved. Any pointers on what the issue may be?
Thanks,
Nitesh
Currently the nexage connector expects the "adm" to contain one macro atleast being the ${AUCTION_PRICE}
// Mandatory Attributes
// Must have adm that includes Nexage macro (at least the price)
configuration_.addField(
"adm",
[](const Json::Value & value, CreativeInfo & data) {
Datacratic::jsonDecode(value, data.adm);
if (data.adm.find("${AUCTION_PRICE}") == std::string::npos) {
throw std::invalid_argument("${AUCTION_PRICE} is expected in adm");
}
return true;
}).snippet();
This is a reminder issue to change this mandatory attribute.
We should just check that it is not empty.
master is version ==?
where can dowload rtbkit 2.0 version?
thanks
Hi,
After pulling the latest files from RTB repository, rtbkit compilation got failed on mock_exchange.cc
/rtbkit/testing/mock_exchange.cc:86:1: error: ‘Stream’ in ‘struct RTBKIT::MockExchange’ does not name a type
./rtbkit/testing/mock_exchange.cc:103:1: error: ‘Stream’ in ‘struct RTBKIT::MockExchange’ does not name a type
./rtbkit/testing/mock_exchange.cc:112:15: error: ‘RTBKIT::MockExchange::Stream’ has not been declared
./rtbkit/testing/mock_exchange.cc: In function ‘void RTBKIT::connect()’:
./rtbkit/testing/mock_exchange.cc:115:8: error: ‘fd’ was not declared in this scope
./rtbkit/testing/mock_exchange.cc:117:5: error: ‘fd’ was not declared in this scope
./rtbkit/testing/mock_exchange.cc:121:33: error: ‘addr’ was not declared in this scope
./rtbkit/testing/mock_exchange.cc: At global scope:
./rtbkit/testing/mock_exchange.cc:132:15: error: ‘RTBKIT::MockExchange::BidStream’ has not been declared
./rtbkit/testing/mock_exchange.cc: In function ‘void RTBKIT::sendBidRequest(const RTBKIT::BidRequest&)’:
./rtbkit/testing/mock_exchange.cc:151:24: error: ‘fd’ was not declared in this scope
./rtbkit/testing/mock_exchange.cc: At global scope:
./rtbkit/testing/mock_exchange.cc:166:15: error: ‘RTBKIT::MockExchange::BidStream’ has not been declared
./rtbkit/testing/mock_exchange.cc: In function ‘std::pair<bool, std::vector<RTBKIT::Bid, std::allocatorRTBKIT::Bid > > RTBKIT::parseResponse(const string&)’:
./rtbkit/testing/mock_exchange.cc:197:13: error: ‘struct RTBKIT::Bid’ has no member named ‘adSpotId’
./rtbkit/testing/mock_exchange.cc:198:13: error: ‘struct RTBKIT::Bid’ has no member named ‘maxPrice’
./rtbkit/testing/mock_exchange.cc: At global scope:
./rtbkit/testing/mock_exchange.cc:208:15: error: ‘RTBKIT::MockExchange::BidStream’ has not been declared
./rtbkit/testing/mock_exchange.cc: In function ‘std::pair<bool, std::vector<RTBKIT::Bid, std::allocatorRTBKIT::Bid > > RTBKIT::recvBid()’:
./rtbkit/testing/mock_exchange.cc:213:20: error: ‘fd’ was not declared in this scope
./rtbkit/testing/mock_exchange.cc: At global scope:
./rtbkit/testing/mock_exchange.cc:228:15: error: ‘RTBKIT::MockExchange::BidStream’ has not been declared
./rtbkit/testing/mock_exchange.cc: In function ‘RTBKIT::BidRequest RTBKIT::makeBidRequest()’:
./rtbkit/testing/mock_exchange.cc:247:31: error: ‘id’ was not declared in this scope
./rtbkit/testing/mock_exchange.cc:247:47: error: ‘key’ was not declared in this scope
./rtbkit/testing/mock_exchange.cc:247:47: note: suggested alternative:
./rtbkit/adgear-api/motrixi/../redislib/redisclient.h:185:10: note: ‘redis::key’
./rtbkit/testing/mock_exchange.cc: At global scope:
./rtbkit/testing/mock_exchange.cc:261:15: error: ‘RTBKIT::MockExchange::WinStream’ has not been declared
./rtbkit/testing/mock_exchange.cc: In function ‘void RTBKIT::sendWin(const RTBKIT::BidRequest&, const RTBKIT::Bid&, const RTBKIT::Amount&)’:
./rtbkit/testing/mock_exchange.cc:267:26: error: ‘const struct RTBKIT::Bid’ has no member named ‘adSpotId’
./rtbkit/testing/mock_exchange.cc:272:30: error: ‘const struct RTBKIT::Bid’ has no member named ‘bidTimestamp’
./rtbkit/testing/mock_exchange.cc:299:24: error: ‘fd’ was not declared in this scope
./rtbkit/testing/mock_exchange.cc:309:11: error: ‘fd’ was not declared in this scope
make: *** [build/x86_64/obj/rtbkit/testing/mock_exchange.a3bd9b6bde15e94ebf5399d38a2148ca.lo] Error 1
Kindly tell me the solution of the issue.
The rtbDataLogger subscription should be optional when running the monitor. It could be the case that you want to run the system without that specific component.
Context:
https://github.com/rtbkit/rtbkit/blob/master/rtbkit/core/monitor/monitor_service_runner.cc#L48
I propose and additional parameter when running the monitor_runner, like --nodatalogger
If you agree I'll push a PR.
Hi, this is the third time I tried installing and building rtbkit on ubuntu 14 lts on AWS. However, I'm failing when running make all
inside platform-deps
folder:
After like 15 minutes:
In file included from ./boost/config.hpp:57:0,
from ./boost/detail/workaround.hpp:41,
from ./boost/test/utils/basic_cstring/basic_cstring_fwd.hpp:19,
from ./boost/test/utils/basic_cstring/basic_cstring.hpp:20,
from ./boost/test/detail/global_typedef.hpp:18,
from ./boost/test/detail/unit_test_parameters.hpp:18,
from ./boost/test/impl/unit_test_parameters.ipp:22,
from libs/test/src/unit_test_parameters.cpp:16:
./boost/lexical_cast.hpp: In member function ‘bool boost::detail::lexical_stream_limited_src<CharT, Traits, RequiresStringbuffer>::shl_char(T)’:
./boost/static_assert.hpp:125:21: warning: typedef ‘boost_static_assert_typedef_1286’ locally defined but not used [-Wunused-local-typedefs]
BOOST_JOIN(boost_static_assert_typedef_, __LINE__)
^
./boost/static_assert.hpp:23:45: note: in expansion of macro ‘BOOST_STATIC_ASSERT’
# define BOOST_STATIC_ASSERT_MSG( B, Msg ) BOOST_STATIC_ASSERT( B )
^
./boost/lexical_cast.hpp:1284:17: note: in expansion of macro ‘BOOST_STATIC_ASSERT_MSG’
BOOST_STATIC_ASSERT_MSG(( sizeof(T) <= sizeof(CharT)) ,
^
./boost/lexical_cast.hpp: In member function ‘bool boost::detail::lexical_stream_limited_src<CharT, Traits, RequiresStringbuffer>::shl_char_array(const T*)’:
./boost/static_assert.hpp:125:21: warning: typedef ‘boost_static_assert_typedef_1311’ locally defined but not used [-Wunused-local-typedefs]
BOOST_JOIN(boost_static_assert_typedef_, __LINE__)
^
./boost/static_assert.hpp:23:45: note: in expansion of macro ‘BOOST_STATIC_ASSERT’
# define BOOST_STATIC_ASSERT_MSG( B, Msg ) BOOST_STATIC_ASSERT( B )
^
./boost/lexical_cast.hpp:1309:17: note: in expansion of macro ‘BOOST_STATIC_ASSERT_MSG’
BOOST_STATIC_ASSERT_MSG(( sizeof(T) <= sizeof(CharT)),
^
./boost/lexical_cast.hpp: In member function ‘bool boost::detail::lexical_stream_limited_src<CharT, Traits, RequiresStringbuffer>::shr_xchar(T&)’:
./boost/static_assert.hpp:125:21: warning: typedef ‘boost_static_assert_typedef_1739’ locally defined but not used [-Wunused-local-typedefs]
BOOST_JOIN(boost_static_assert_typedef_, __LINE__)
^
./boost/static_assert.hpp:23:45: note: in expansion of macro ‘BOOST_STATIC_ASSERT’
# define BOOST_STATIC_ASSERT_MSG( B, Msg ) BOOST_STATIC_ASSERT( B )
^
./boost/lexical_cast.hpp:1737:17: note: in expansion of macro ‘BOOST_STATIC_ASSERT_MSG’
BOOST_STATIC_ASSERT_MSG(( sizeof(CharT) == sizeof(T) ),
^
gcc.archive bin.v2/libs/test/build/gcc-4.8/release/link-static/threading-multi/libboost_test_exec_monitor.a
common.copy /home/ubuntu/local/lib/libboost_test_exec_monitor.a
...failed updating 2 targets...
...skipped 6 targets...
...updated 10322 targets...
make: *** [install_boost] Error 1
(these are last few lines)
After this I tried:
sudo apt-get install libboost-all-dev
then I get:
/usr/include/x86_64-linux-gnu/unicode/numfmt.h:333:20: note: icu_52::UnicodeString& icu_52::NumberFormat::format(int32_t, icu_52::UnicodeString&) const
UnicodeString& format( int32_t number,
^
/usr/include/x86_64-linux-gnu/unicode/numfmt.h:346:20: note: icu_52::UnicodeString& icu_52::NumberFormat::format(int64_t, icu_52::UnicodeString&) const
UnicodeString& format( int64_t number,
^
"g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_CHRONO_DYN_LINK=1 -DBOOST_LOCALE_DYN_LINK=1 -DBOOST_LOCALE_NO_WINAPI_BACKEND=1 -DBOOST_LOCALE_WITH_ICONV=1 -DBOOST_LOCALE_WITH_ICU=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_THREAD_BUILD_DLL=1 -DBOOST_THREAD_NO_LIB=1 -DBOOST_THREAD_POSIX -DBOOST_THREAD_USE_DLL=1 -DNDEBUG -I"." -c -o "bin.v2/libs/locale/build/gcc-4.8/release/threading-multi/icu/formatter.o" "libs/locale/src/icu/formatter.cpp"
...failed gcc.compile.c++ bin.v2/libs/locale/build/gcc-4.8/release/threading-multi/icu/formatter.o...
...skipped <pbin.v2/libs/locale/build/gcc-4.8/release/threading-multi>libboost_locale.so.1.52.0 for lack of <pbin.v2/libs/locale/build/gcc-4.8/release/threading-multi>icu/formatter.o...
...skipped <p/home/ubuntu/local/lib>libboost_locale.so.1.52.0 for lack of <pbin.v2/libs/locale/build/gcc-4.8/release/threading-multi>libboost_locale.so.1.52.0...
...skipped <p/home/ubuntu/local/lib>libboost_locale.so for lack of <p/home/ubuntu/local/lib>libboost_locale.so.1.52.0...
...failed updating 2 targets...
...skipped 6 targets...
make: *** [install_boost] Error 1
HELP!
Hello guys, looks like this lambda
/* This lambda will get called when the post auction loop receives a win
on an auction.
*/
palEvents.messageHandler = [&] (const vector<zmq::message_t>& msg)
{
RTBKIT::AccountKey account(msg[19].toString());
RTBKIT::UserIds uids =
RTBKIT::UserIds::createFromJson(msg[15].toString());
storage->inc(account, uids);
recordHit("wins");
};
from augmentor example never called. Don't know why.
Should we init palEvents
before subscribing?
After a clean install (virtualized Ubuntu Server 12.04.02), when running make test
, the job stalls at [TESTCASE] multiple_service_test
. There's no outstanding active process running in the machine, with the occasional bump in CPU from zookeeper
.
for example ,video in Youtube, or architecture blogs?
this is a amazing rtb system, but the foundation of skill is diffcult.
nodejs, cpp, redis,zookeeper.
You may be can publish a ebook (pdf best) first.
Hi,
I am building a mobile connector. I am getting requests in Open RTB Mobile JSON formats. If I send those to directly to the core kit, it is giving me exceptions.
So this is my question. Does the core kit support mobile RTB? Or the expectation is we should transform the mobile RTB request into regular RTB with a few hacks and the receiving response needs to be converted back to mobile format.
Thanks,
ma
Hi after 2 day warfight I start this topic.
Thanks for help.
/// EDIT
https://groups.google.com/a/rtbkit.org/forum/#!topic/discuss/v4Lx-dOVBU8
/home/ubuntu/local/include/boost/random/detail/integer_log2.hpp:59:35: error: always_inline function might not be inlinable [-Werror=attributes]
Please help in this regard too.
The https://github.com/rtbkit/rtbkit/wiki/Standard-Ad-Server-Protocol currently includes "userIds" field, that is just ignored by the standard ad server connector.
Also, the Post Auction Loop seems not to use that field in any meaningful way.
Something probably should happen - some code should be added to the connector to parse the userIds, or else this field should be removed. I opt for the second solution, as getting access to the bid request's userIds in the ad server is nontrivial... Or they aren't required to be the same ids?
(Also, why the dataCost is transferred in the the win notification, and not simply returned by the bidder, who knows which data he used?)
I am getting error in make compile NODEJS_ENABLED=0 steps. I am using ubuntu 14.04.
[PBUF c++] rtbkit/plugins/exchange/realtime-bidding.proto
/bin/bash: protoc: command not found
make: *** [build/x86_64/gen/rtbkit/plugins/exchange/realtime-bidding.pb.cc] Error 127
Hi, I am trying to run the python http bidder as described in the wiki!
My http-config.json
is exactly as in the examples folder..
My sample.http_bidder_config.json
:
{
"type": "http",
"router": {
"host": "http://localhost:7654",
"path": "/"
},
"adserver": {
"host": "http://localhost",
"winPort": 7653,
"eventPort": 7652
}
}
When I run python http_bid_agent_ex.py
inside the py-bidder folder: I get:
ACS reg'ing: {"bidderInterface": "iface.http", "bidProbability": 0.5, "account": ["hello", "world"], "externalId": 123, "creatives": [{"width": 300, "id": 1, "height": 250}, {"width": 300, "id": 2, "height": 250}]}
Budgeting: {"USD/1M": 500000}
WARNING:root:Connect error on fd 13: ECONNREFUSED
Request Error!
Request response OK
Budgeting: {"USD/1M": 500000}
WARNING:root:Connect error on fd 12: ECONNREFUSED
Request Error!
My sample.http_bidder-config.json looks like:
......
{
"name": "router",
"root": ".",
"path": "build/x86_64/bin/router_runner",
"arg": [
"-B", "rtbkit/sample.bootstrap.json",
"--bidder", "rtbkit/sample.http_bidder-config.json"
],
"log": true
},
{
"name": "post-auction",
"root": ".",
"path": "build/x86_64/bin/post_auction_runner",
"arg": [
"-B", "rtbkit/sample.bootstrap.json",
"--win-seconds", "3600.0",
"--auction-seconds", "900.0",
"--bidder", "rtbkit/sample.http_bidder-config.json"
],
"log": true
},
......
Firstly, do I have to move any file(s) when running?
Does order matter? Can I python http_bid_agent_ex.py
before running ./lauch.sh
All I see in the output is(even after budgeting):
rtbAgentConfiguration:
agentConfigurationService OK 2016-06-26 04:59:12 Alive
rtbBanker:
PostAuctionLoop.slaveBanker OK 2016-06-26 04:59:12 Sync with MasterBanker: OK
router.slaveBanker OK 2016-06-26 04:59:12 Sync with MasterBanker: OK
rtbDataLogger:
data_logger OK 2016-06-26 04:59:12 Alive
rtbPostAuctionService:
PostAuctionLoop ERR 2016-06-26 04:59:12 WinLoss pipe: ERROR, CampaignEvent pipe: ERROR, Banker: OK
rtbRequestRouter:
router OK 2016-06-26 04:59:12 Connection to PAL: OK, Banker: OK
Can the build be made on Amazon Linux/RHEL?
Hi!
Today i've found data_logger process crashed with unhandled exception.
Last lines from stderr:
2013-11-15 18:19:16,352:11890(0x7f0071d24700):ZOO_ERROR@handle_socket_error_msg@
1721: Socket [127.0.0.1:2181] zk retcode=-4, errno=112(Host is down): failed whi
le receiving a server response
2013-11-15 18:19:16,362:11890(0x7f0071d24700):ZOO_ERROR@handle_socket_error_msg@
1739: Socket [127.0.0.1:2181] zk retcode=-112, errno=116(Stale NFS file handle):
sessionId=0x2424c7324bc0097 has expired.
terminate called after throwing an instance of 'ML::Exception'
what(): can't connect; handle already exists
Back trace of two threads from core dump:
(gdb) thread 1
[Switching to thread 1 (Thread 0x7f0071523700 (LWP 11902))]
#0 0x00007f007e860425 in __GI_raise (sig=)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 in ../nptl/sysdeps/unix/sysv/linux/raise.c
(gdb) bt
#0 0x00007f007e860425 in __GI_raise (sig=)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007f007e863b8b in __GI_abort () at abort.c:91
#2 0x00007f007f15bb05 in __gnu_cxx::__verbose_terminate_handler() ()
from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007f007f159c76 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007f007f159ca3 in std::terminate() ()
from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007f007f159ece in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007f0080bb7b33 in Datacratic::ZookeeperConnection::connectImpl (
this=this@entry=0x2547150, host=..., timeoutInSeconds=5,
timeoutInSeconds@entry=<error reading variable: Could not find type for DW_OP_GNU_const_type>, clientId=clientId@entry=0x0) at ./soa/service/zookeeper.cc:183
#7 0x00007f0080bb7b5c in Datacratic::ZookeeperConnection::connect (
this=this@entry=0x2547150, host=...,
timeoutInSeconds=<error reading variable: Could not find type for DW_OP_GNU_const_type>) at ./soa/service/zookeeper.cc:217
#8 0x00007f0080bb87a2 in Datacratic::ZookeeperConnection::reconnect (
this=this@entry=0x2547150) at ./soa/service/zookeeper.cc:247
#9 0x00007f0080bb89ac in Datacratic::ZookeeperConnection::checkRes (
this=this@entry=0x2547150, returnCode=-112, retries=@0x7f007152285c: 0,
operation=operation@entry=0x7f0080bc9ce9 "zoo_get_children",
path=0x7f00640164b8 "/tddsp-prod/serviceClass/monitor")
at ./soa/service/zookeeper.cc:283
#10 0x00007f0080bb90b9 in Datacratic::ZookeeperConnection::getChildren (this=0x2547150,
path_=..., failIfNodeMissing=true,
watcher=0x7f0080b8f1b0 <Datacratic::watcherFn(int, int, std::string const&, void*)>, watcherData=0x7f0064015bb0) at ./soa/service/zookeeper.cc:575
#11 0x00007f0080b8ee95 in Datacratic::ZookeeperConfigurationService::getChildren (
this=this@entry=0x2547120, key=..., watch=...)
at ./soa/service/zookeeper_configuration_service.cc:217
#12 0x00007f0080ba3cb4 in Datacratic::MultiRestProxy::onServiceProvidersChanged (
this=0x7fffbd6ed520, path=..., local=local@entry=true)
at ./soa/service/rest_proxy.cc:340
#13 0x00007f0080ba48cb in operator() (__closure=0x2547900)
at ./soa/service/rest_proxy.cc:309
#14 std::_Function_handler<void(std::basic_string<char, std::char_traits, std::allo---Type to continue, or q to quit---
cator >, Datacratic::ConfigurationService::ChangeType), Datacratic::MultiRestProxy::connectAllServiceProviders(const string&, const string&, bool)::<lambda(const string&, Datacratic::ConfigurationService::ChangeType)> >::_M_invoke(const std::_Any_data &, std::basic_string<char, std::char_traits, std::allocator >, Datacratic::ConfigurationService::ChangeType) (__functor=..., __args#0=..., __args#1=)
at /usr/include/c++/4.7/functional:1925
#15 0x00007f0080b8f240 in operator() (
__args#1=Datacratic::ConfigurationService::CREATED, __args#0=...,
this=<optimized out>) at /usr/include/c++/4.7/functional:2310
#16 Datacratic::watcherFn (type=, state=, path=...,
watcherCtx=<optimized out>) at ./soa/service/zookeeper_configuration_service.cc:102
#17 0x00007f0080bb7fe6 in call (state=1, type=-1, this=0x7f0064000a70)
at ./soa/service/zookeeper.h:97
#18 Datacratic::(anonymous namespace)::zk_callback (ah=, type=-1,
state=1, path=<optimized out>, user=<optimized out>)
at ./soa/service/zookeeper.cc:35
#19 0x00007f007c064d91 in do_foreach_watcher (state=1, type=-1, path=0x7f0064000920 "",
zh=0x2552ea0, wo=0x7f006c000920) at src/zk_hashtable.c:279
#20 deliverWatchers (zh=0x2552ea0, type=-1, state=1, path=0x7f0064000920 "",
list=0x7f006c002de0) at src/zk_hashtable.c:321
#21 0x00007f007c05a50d in process_completions (zh=0x2552ea0) at src/zookeeper.c:2108
#22 0x00007f007c065231 in do_completion (v=0x2552ea0) at src/mt_adaptor.c:466
#23 0x00007f007e614e9a in start_thread (arg=0x7f0071523700) at pthread_create.c:308
#24 0x00007f007e91dccd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#25 0x0000000000000000 in ?? ()
(gdb) thread 9
[Switching to thread 9 (Thread 0x7f00488f7700 (LWP 11930))]
#0 pthread_cond_timedwait@@GLIBC_2.3.2 ()
at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:215
215 ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S: No such file or directory.
(gdb) bt
#0 pthread_cond_timedwait@@GLIBC_2.3.2 ()
at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:215
#1 0x00007f0080bb77d4 in __gthread_cond_timedwait (__abs_timeout=0x7f00488f65b0,
__mutex=0x2547150, __cond=0x2547178)
at /usr/include/x86_64-linux-gnu/c++/4.7/./bits/gthr-default.h:886
#2 __wait_until_impl<std::chrono::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000l> > > (__atime=..., __lock=, this=0x2547178)
at /usr/include/c++/4.7/condition_variable:164
#3 wait_until<std::chrono::duration<long, std::ratio<1l, 1000000l> > > (__atime=...,
__lock=<synthetic pointer>, this=0x2547178)
at /usr/include/c++/4.7/condition_variable:100
#4 wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=,
this=0x2547178) at /usr/include/c++/4.7/condition_variable:132
#5 Datacratic::ZookeeperConnection::connectImpl (this=this@entry=0x2547150, host=...,
timeoutInSeconds=5,
timeoutInSeconds@entry=<error reading variable: Could not find type for DW_OP_GNU_const_type>, clientId=clientId@entry=0x0) at ./soa/service/zookeeper.cc:197
#6 0x00007f0080bb7b5c in Datacratic::ZookeeperConnection::connect (
this=this@entry=0x2547150, host=...,
timeoutInSeconds=<error reading variable: Could not find type for DW_OP_GNU_const_type>) at ./soa/service/zookeeper.cc:217
#7 0x00007f0080bb87a2 in Datacratic::ZookeeperConnection::reconnect (
this=this@entry=0x2547150) at ./soa/service/zookeeper.cc:247
#8 0x00007f0080bb89ac in Datacratic::ZookeeperConnection::checkRes (
this=this@entry=0x2547150, returnCode=-112, retries=@0x7f00488f66fc: 0,
operation=operation@entry=0x7f0080bc9ce9 "zoo_get_children",
path=0x7f0020014f98 "/tddsp-prod/serviceClass/rtbRequestRouter")
at ./soa/service/zookeeper.cc:283
#9 0x00007f0080bb90b9 in Datacratic::ZookeeperConnection::getChildren (this=0x2547150,
path_=..., failIfNodeMissing=true,
watcher=0x7f0080b8f1b0 <Datacratic::watcherFn(int, int, std::string const&, void*)>, watcherData=0x7f0020000980) at ./soa/service/zookeeper.cc:575
#10 0x00007f0080b8ee95 in Datacratic::ZookeeperConfigurationService::getChildren (
this=this@entry=0x2547120, key=..., watch=...)
at ./soa/service/zookeeper_configuration_service.cc:217
#11 0x00007f0080e19e25 in Datacratic::ServiceProviderWatcher::handleServiceClassChange (
this=<optimized out>, serviceClass=...) at ./soa/service/zmq_named_pub_sub.h:858
---Type to continue, or q to quit---
#12 0x00007f0080e1b264 in Datacratic::TypedMessageSinkstd::string::processOne (
this=0x7fffbd6ed418) at ./soa/service/typed_message_channel.h:73
#13 0x00007f0080b84808 in Datacratic::MessageLoop::handleEpollEvent (
this=<optimized out>, event=...) at ./soa/service/message_loop.cc:346
#14 0x00007f0080b73d4a in operator() (__args#0=..., this=0x7fffbd6ed150)
at /usr/include/c++/4.7/functional:2310
#15 Datacratic::Epoller::handleEvents(int, int, std::function<bool (epoll_event&)> const&, std::function<void ()> const&, std::function<void ()> const&) (this=0x7fffbd6ed130,
usToWait=0, nEvents=1, handleEvent_=..., beforeSleep_=..., afterSleep_=...)
at ./soa/service/epoller.cc:180
#16 0x00007f0080b5890b in Datacratic::Epoller::processOne (this=0x7fffbd6ed130)
at ./soa/service/epoller.h:85
#17 0x00007f0080b86a33 in Datacratic::MessageLoop::processOne (this=0x7fffbd6eceb8)
at ./soa/service/message_loop.cc:385
#18 0x00007f0080b84af9 in Datacratic::MessageLoop::runWorkerThread (this=0x7fffbd6eceb8)
at ./soa/service/message_loop.cc:293
#19 0x00007f0080b850b8 in operator() (__closure=0x25581d8)
at ./soa/service/message_loop.cc:84
#20 boost::detail::thread_dataDatacratic::MessageLoop::start(std::function<void())::<lambda()> >::run(void) (this=0x2558020)
at /home/dsp/local/include/boost/thread/detail/thread.hpp:117
#21 0x00007f007dc726da in thread_proxy ()
from /home/dsp/local/lib/libboost_thread.so.1.53.0
#22 0x00007f007e614e9a in start_thread (arg=0x7f00488f7700) at pthread_create.c:308
#23 0x00007f007e91dccd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#24 0x0000000000000000 in ?? ()
If i understand correctly, two threads started reconnect to Zookeeper, one of them created new handle and second threw exception "handle already exists".
rtbkit@ip-10-67-173-116:~/rtbkit$ make compile [NODEJS_ADDON] libbid_request_node_impl.so /usr/bin/ld: cannot find -ljs collect2: ld returned 1 exit status make: *** [build/x86_64/bin/libbid_request_node_impl.f85de100f3b4cb7113227646b1565104.so] Error 1
Ubuntu version 12.04
how can i install this lib ?
please tell me, thanks
Does anybody know if for learning purposes only is posible to compile and run RTB Kit on MAC OS X El Captain version ?
If so, what are the enviroment setup to compile the RTB kit ?
Thanks.
Hello guys. Seems that if I launch demo stack as described in wiki in this order:
./build/x86_64/bin/mock_exchange_runner
./build/x86_64/bin/launcher --node localhost --script ./launch.sh sample.launch.json
router failed with exception:
connecting to masterBanker/zeromq
connected to tcp://127.0.0.1:22000
zookeeper error on zoo_create, path /rtb-test/serviceClass/rtbRequestRouter/router: node exists
----------------- Exception thrown ------------------------
type: ML::Exception
pid: 482; tid: 482
what: Zookeeper error on zoo_create, path /rtb-test/serviceClass/rtbRequestRouter/router: node exists
stack:
00: 0x0x7f949bcb5e07 at __cxa_throw + 0x27 in build/x86_64/bin/libexception_hook.so + 0xe07build/x86_64/bin/libexception_hook.so(__cxa_throw+0x27) [0x7f949bcb5e07]
01: 0x0x7f949afb51d3 at Datacratic::ZookeeperConnection::checkRes(int, int&, char const*, char const*) + 0xe3 in build/x86_64/bin/libservices.so + 0xa81d3build/x86_64/bin/libservices.so(_ZN10Datacratic19ZookeeperConnection8checkResEiRiPKcS3_+0xe3) [0x7f949afb51d3]
02: 0x0x7f949afb5ef9 at Datacratic::ZookeeperConnection::createNode(std::string const&, std::string const&, bool, bool, bool, bool) + 0x439 in build/x86_64/bin/libservices.so + 0xa8ef9build/x86_64/bin/libservices.so(_ZN10Datacratic19ZookeeperConnection10createNodeERKSsS2_bbbb+0x439) [0x7f949afb5ef9]
03: 0x0x7f949af978b5 at Datacratic::ZookeeperConfigurationService::setUnique(std::string const&, Json::Value const&) + 0x285 in build/x86_64/bin/libservices.so + 0x8a8b5build/x86_64/bin/libservices.so(_ZN10Datacratic29ZookeeperConfigurationService9setUniqueERKSsRKN4Json5ValueE+0x285) [0x7f949af978b5]
04: 0x0x7f949af86a2d at Datacratic::ServiceBase::registerServiceProvider(std::string const&, std::vector<std::string, std::allocator<std::string> > const&) + 0x17d in build/x86_64/bin/libservices.so + 0x79a2dbuild/x86_64/bin/libservices.so(_ZN10Datacratic11ServiceBase23registerServiceProviderERKSsRKSt6vectorISsSaISsEE+0x17d) [0x7f949af86a2d]
05: 0x0x7f9498949ef2 at RTBKIT::Router::init() + 0xd2 in build/x86_64/bin/librtb_router.so + 0x5bef2build/x86_64/bin/librtb_router.so(_ZN6RTBKIT6Router4initEv+0xd2) [0x7f9498949ef2]
06: 0x0x7f949b460dda at RTBKIT::RouterRunner::init() + 0x43a in build/x86_64/bin/librouter_runner.so + 0x17ddabuild/x86_64/bin/librouter_runner.so(_ZN6RTBKIT12RouterRunner4initEv+0x43a) [0x7f949b460dda]
07: 0x0x40e05a at main + 0xaa in build/x86_64/bin/router_ex + 0xe05abuild/x86_64/bin/router_ex(main+0xaa) [0x40e05a]
08: 0x0x7f949a3ea30d at __libc_start_main + 0xed in /lib/x86_64-linux-gnu/libc.so.6 + 0x2130d/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f949a3ea30d]
09: 0x0x40e549 in build/x86_64/bin/router_ex + 0xe549build/x86_64/bin/router_ex() [0x40e549]
terminate called after throwing an instance of 'ML::Exception'
what(): Zookeeper error on zoo_create, path /rtb-test/serviceClass/rtbRequestRouter/router: node exists
Though if I launch in another order:
./build/x86_64/bin/launcher --node localhost --script ./launch.sh sample.launch.json
./build/x86_64/bin/mock_exchange_runner
the launch is successful. I do not think that this is a serious problem, but still unpleasant
While linking to exchange.so I had this error:
~/sources/daftcodertb/rtbkit/build/x86_64/bin/libexchange.so: undefined reference to `Datacratic::GcLockBase::exitCS(Datacratic::GcLockBase::ThreadGcInfoEntry*, Datacratic::GcLockBase::RunDefer)'
Adding gc to LIBRTB_EXCHANGE_LINK in rtbkit/plugins/exchange/exchange.mk fixes the problem
A creative macro for should be added to /common/creative_configuration.h
that reports the values of installation
and location
from bootstrap.json
.
Adding these would simplify reporting win events to the correct PAL when using a multi-datacenter setup.
I'd be happy to submit a pull request if this is something that will be merged.
Hi, i follow the instruction from here https://github.com/rtbkit/rtbkit/wiki/RTBkit-Deps-and-Ubuntu-14.04 but get a bunch of error when trying to compile rtbkit in ubuntu 14.04.
Form.cpp:(.text+0x295): undefined reference to curl_formfree' /home/ubuntu/Documents/workspace/rtbkit-deps/curlpp/src/curlpp/.libs/libcurlpp.a(Form.o): In function
curlpp::FormParts::File::add(curl_httppost**, curl_httppost**)':
Form.cpp:(.text+0x916): undefined reference to curl_formadd' Form.cpp:(.text+0x98b): undefined reference to
curl_formadd'
/home/ubuntu/Documents/workspace/rtbkit-deps/curlpp/src/curlpp/.libs/libcurlpp.a(Form.o): In function curlpp::FormParts::Content::add(curl_httppost**, curl_httppost**)': Form.cpp:(.text+0xdf2): undefined reference to
curl_formadd'
Form.cpp:(.text+0xe67): undefined reference to `curl_formadd'
ollect2: error: ld returned 1 exit status
make[2]: *** [example22] Error 1
make[2]: *** [example21] Error 1
/home/ubuntu/Documents/workspace/rtbkit-deps/curlpp/src/curlpp/.libs/libcurlpp.a(cURLpp.o): In function curlpp::initialize(long)': cURLpp.cpp:(.text+0x14): undefined reference to
curl_global_init'
/home/ubuntu/Documents/workspace/rtbkit-deps/curlpp/src/curlpp/.libs/libcurlpp.a(cURLpp.o): In function curlpp::terminate()': cURLpp.cpp:(.text+0x31): undefined reference to
curl_global_cleanup'
/home/ubuntu/Documents/workspace/rtbkit-deps/curlpp/src/curlpp/.libs/libcurlpp.a(cURLpp.o): In function curlpp::escape(std::string const&)': cURLpp.cpp:(.text+0x99): undefined reference to
curl_escape'
cURLpp.cpp:(.text+0x12a): undefined reference to curl_free' Form.cpp:(.text+0xdf2): undefined reference to
curl_formadd'
Form.cpp:(.text+0xe67): undefined reference to curl_formadd' collect2: error: ld returned 1 exit status make[2]: *** [example18] Error 1 make[2]: Target
all' not remade because of errors.
make[2]: Leaving directory /home/ubuntu/Documents/workspace/rtbkit-deps/curlpp/examples' Making all in doc make[2]: Entering directory
/home/ubuntu/Documents/workspace/rtbkit-deps/curlpp/doc'
make[2]: Nothing to be done for all'. make[2]: Leaving directory
/home/ubuntu/Documents/workspace/rtbkit-deps/curlpp/doc'
make[2]: Entering directory /home/ubuntu/Documents/workspace/rtbkit-deps/curlpp' make[2]: Nothing to be done for
all-am'.
make[2]: Leaving directory /home/ubuntu/Documents/workspace/rtbkit-deps/curlpp' make[1]: *** [all-recursive] Error 1 make[1]: Target
all' not remade because of errors.
make[1]: Leaving directory `/home/ubuntu/Documents/workspace/rtbkit-deps/curlpp'
make: *** [install_curlpp] Error 2
can some one enlighten me which can be done to solve the problem? It seems that there is some problem with libcurl? Thanks.
Hello guys! I have a question about OpenRTB::Video object. Looks like there are some discrepancy between RTBKIT implementation and OpenRTB 2.1 standart. If you look to the structure of the Video object in standart specification you will see that protocol field is defined as integer, but we have an List in OpenRTB::Video structure.
However there is an example in OpenRTB specification and protocol field contains a list there:
“protocol” : [1,2,3,4],
One of our partners sending protocol field as an integer, so default OpenRTB parser from RTBKIT failed to parse such requests.
So I'm curious is it an issue in OpenRTB specification, or in RTBKIT implementation?
Hi!
I've got segfault with such stack trace:
#0 0x00007f368985f141 in Datacratic::watcherFn (type=-1, path=..., watcherCtx=0x1abb820) at ./soa/service/zookeeper_configuration_service.cc:75
#1 0x00007f3689884e94 in call (type=-1, this=0x1ac00f0) at ./soa/service/zookeeper.h:72
#2 Datacratic::(anonymous namespace)::zk_callback (ah=, type=-1, state=, path=, user=0x1ac00f0) at ./soa/service/zookeeper.cc:27
#3 0x00007f368695fd91 in do_foreach_watcher (state=3, type=-1, path=0x7f3670000940 "", zh=0x1abdf40, wo=0x7f36780020f0) at src/zk_hashtable.c:279
#4 deliverWatchers (zh=0x1abdf40, type=-1, state=3, path=0x7f3670000940 "", list=0x7f3678002260) at src/zk_hashtable.c:321
#5 0x00007f368695550d in process_completions (zh=0x1abdf40) at src/zookeeper.c:2108
#6 0x00007f3686960231 in do_completion (v=0x1abdf40) at src/mt_adaptor.c:466
#7 0x00007f36887bee9a in start_thread (arg=0x7f367cd4b700) at pthread_create.c:308
#8 0x00007f3688ac7ccd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#9 0x0000000000000000 in ?? ()
It appears when some of the rtbkit components loses connection with ZooKeeper.
Hi!
Next code in AugmentationLoop::doDisconnection() method looks like a bug:
for (auto it = augmentor.instances.begin(),
end = augmentor.instances.end();
it != end; ++it)
{
if (it->addr != addr) continue;
augmentor.instances.erase(it);
recordHit("augmentor.%s.instances.%s.disconnected",
augmentor.name, it->addr);
break;
}
After erasing iterator it becomes invalidated and cannot be dereferenced (it->addr).
When receiving a big number as part of the wseat array, RTBkit tries to convert it into int and there's an integer overflow as a result.
Some ids in the wseat array are big enough for producing this problem.
The code can be found here:
rtbkit/rtbkit/common/segments.h
Line 78 in b67f5c4
and here:
rtbkit/rtbkit/common/segments.cc
Line 364 in b67f5c4
I solved it using unsigned long int instead as a quick fix, but probably should treat as a string.
hi,
i was trying to make and install the rtb-kit on a fresh install of ubuntu 12.04 lts
i was able to make it successfully
but 'make test' failed with the error " http_long_header_test FAILED"
is there anything i am missing to install ? kindly assist.
the detailed output from screen is below:
~/rtbkit-preview# make test
[TESTCASE] http_long_header_test
http_long_header_test FAILED
Running 1 test case...
listening on hostname localhost port 15000
port = 15000
----------------- Exception thrown ------------------------
type: ML::Exception
pid: 27132; tid: 27134
what: HTTP header exceeds 16kb
stack:
00: 0x0x2b034e184df7 at __cxa_throw + 0x27 in build/x86_64/bin/libexception_hook.so + 0xdf7build/x86_64/bin/libexception_hook.so(__cxa_throw+0x27) [0x2b034e184df7]
01: 0x0x2b034f661346 at Datacratic::HttpConnectionHandler::handleData(std::string const&) + 0x346 in build/x86_64/bin/libservices.so + 0x5f346build/x86_64/bin/libservices.so(_ZN10Datacratic21HttpConnectionHandler10handleDataERKSs+0x346) [0x2b034f661346]
02: 0x0x2b034f66007d at Datacratic::PassiveConnectionHandler::handleInput() + 0x2fd in build/x86_64/bin/libservices.so + 0x5e07dbuild/x86_64/bin/libservices.so(_ZN10Datacratic24PassiveConnectionHandler11handleInputEv+0x2fd) [0x2b034f66007d]
03: 0x0x2b034f653ef6 at Datacratic::TransportBase::handleInput() + 0xe6 in build/x86_64/bin/libservices.so + 0x51ef6build/x86_64/bin/libservices.so(_ZN10Datacratic13TransportBase11handleInputEv+0xe6) [0x2b034f653ef6]
04: 0x0x2b034f650a57 at Datacratic::TransportBase::handleEvents() + 0x6b7 in build/x86_64/bin/libservices.so + 0x4ea57build/x86_64/bin/libservices.so(_ZN10Datacratic13TransportBase12handleEventsEv+0x6b7) [0x2b034f650a57]
05: 0x0x2b034f656f56 at Datacratic::EndpointBase::handleEpollEvent(epoll_event&) + 0x46 in build/x86_64/bin/libservices.so + 0x54f56build/x86_64/bin/libservices.so(_ZN10Datacratic12EndpointBase16handleEpollEventER11epoll_event+0x46) [0x2b034f656f56]
06: 0x0x2b034f6720db at Datacratic::Epoller::handleEvents(int, int, std::function<bool (epoll_event&)> const&, std::function<void ()> const&, std::function<void ()> const&) + 0x29b in build/x86_64/bin/libservices.so + 0x700dbbuild/x86_64/bin/libservices.so(ZN10Datacratic7Epoller12handleEventsEiiRKSt8functionIFbR11epoll_eventEERKS1_IFvvEESB+0x29b) [0x2b034f6720db]
07: 0x0x2b034f659b50 at Datacratic::EndpointBase::runEventThread(int, int) + 0x210 in build/x86_64/bin/libservices.so + 0x57b50build/x86_64/bin/libservices.so(_ZN10Datacratic12EndpointBase14runEventThreadEii+0x210) [0x2b034f659b50]
08: 0x0x2b034fb09da9 in /root/local/lib/libboost_thread.so.1.47.0 + 0xdda9/root/local/lib/libboost_thread.so.1.47.0(+0xdda9) [0x2b034fb09da9]
09: 0x0x2b034df6de9a in /lib/x86_64-linux-gnu/libpthread.so.0 + 0x7e9a/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a) [0x2b034df6de9a]
10: 0x0x2b034f336cbd at clone + 0x6d in /lib/x86_64-linux-gnu/libc.so.6 + 0xf3cbd/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x2b034f336cbd]
got error handler input had exception HTTP header exceeds 16kb
http_long_header_test FAILED
make: *** [build/x86_64/tests/http_long_header_test.passed] Error 1
AsyncConnection::EventLoop::run thread consumes 100% CPU, this happend because of unhandled POLLERR flag in revents.
strace shows repeating records:
poll([{fd=67, events=POLLIN}, {fd=66, events=0}], 2, 1000000) = 1 ([{fd=66, revents=POLLERR|POLLHUP}])
This means, I believe, that Redis closed connection, and main cycle of AsyncConnection spins very fast, not handling this state.
I think, we should handle POLLERR state, stop main cycle and close AsyncConnection with appropriate error.
post auction service crashes with various exceptions:
---------------- Exception thrown ------------------------
type: ML::Exception
pid: 28540; tid: 28549
what: attempt to re-open commitment
stack:
00: 0x0x7f554001de07 at __cxa_throw + 0x27 in build/x86_64/bin/libexception_hook.so + 0xe07build/x86_64/bin/libexception_hook.so(__cxa_throw+0x27) [0x7f554001de07]
01: 0x0x7f553f66bfbf at RTBKIT::SlaveBanker::attachBid(RTBKIT::AccountKey const&, std::string const&, RTBKIT::Amount) + 0x1df in build/x86_64/bin/libbanker.so + 0x48fbfbuild/x86_64/bin/libbanker.so(_ZN6RTBKIT11SlaveBanker9attachBidERKNS_10AccountKeyERKSsNS_6AmountE+0x1df) [0x7f553f66bfbf]
02: 0x0x7f553fbde71d at RTBKIT::PostAuctionLoop::doAuction(RTBKIT::SubmittedAuctionEvent const&) + 0x4dd in build/x86_64/bin/libpost_auction.so + 0x3d71dbuild/x86_64/bin/libpost_auction.so(_ZN6RTBKIT15PostAuctionLoop9doAuctionERKNS_21SubmittedAuctionEventE+0x4dd) [0x7f553fbde71d]
03: 0x0x7f553fbdecb7 at RTBKIT::PostAuctionLoop::doAuctionMessage(std::vector<std::string, std::allocator<std::string> > const&) + 0x177 in build/x86_64/bin/libpost_auction.so + 0x3dcb7build/x86_64/bin/libpost_auction.so(_ZN6RTBKIT15PostAuctionLoop16doAuctionMessageERKSt6vectorISsSaISsEE+0x177) [0x7f553fbdecb7]
04: 0x0x4229de at Datacratic::ZmqMessageRouter::handleMessage(std::vector<std::string, std::allocator<std::string> > const&) + 0x10e in build/x86_64/bin/post_auction_runner + 0x229debuild/x86_64/bin/post_auction_runner(_ZN10Datacratic16ZmqMessageRouter13handleMessageERKSt6vectorISsSaISsEE+0x10e) [0x4229de]
05: 0x0x41cff1 at Datacratic::ZmqNamedEndpoint::handleRawMessage(std::vector<zmq::message_t, std::allocator<zmq::message_t> >&&) + 0x1e1 in build/x86_64/bin/post_auction_runner + 0x1cff1build/x86_64/bin/post_auction_runner(_ZN10Datacratic16ZmqNamedEndpoint16handleRawMessageEOSt6vectorIN3zmq9message_tESaIS3_EE+0x1e1) [0x41cff1]
06: 0x0x7f553f94fcab at Datacratic::ZmqBinaryEventSource::processOne() + 0x16b in build/x86_64/bin/libservices.so + 0x9ecabbuild/x86_64/bin/libservices.so(_ZN10Datacratic20ZmqBinaryEventSource10processOneEv+0x16b) [0x7f553f94fcab]
07: 0x0x7f553f93d6dd at Datacratic::MessageLoop::processOne() + 0x11d in build/x86_64/bin/libservices.so + 0x8c6ddbuild/x86_64/bin/libservices.so(_ZN10Datacratic11MessageLoop10processOneEv+0x11d) [0x7f553f93d6dd]
----------------- Exception thrown ------------------------
type: ML::Exception
pid: 27923; tid: 27937
what: TimeoutMap: attempt to re-insert existing key
stack:
00: 0x0x7fd43f625e07 at __cxa_throw + 0x27 in build/x86_64/bin/libexception_hook.so + 0xe07build/x86_64/bin/libexception_hook.so(__cxa_throw+0x27) [0x7fd43f625e07]
01: 0x0x7fd43f1ff076 at Datacratic::TimeoutMap<std::pair<Datacratic::Id, Datacratic::Id>, RTBKIT::FinishedInfo>::doThrowException(std::string const&) const + 0x76 in build/x86_64/bin/libpost_auction.so + 0x56076build/x86_64/bin/libpost_auction.so(_ZNK10Datacratic10TimeoutMapISt4pairINS_2IdES2_EN6RTBKIT12FinishedInfoEE16doThrowExceptionERKSs+0x76) [0x7fd43f1ff076]
02: 0x0x7fd43f204d1d at Datacratic::TimeoutMap<std::pair<Datacratic::Id, Datacratic::Id>, RTBKIT::FinishedInfo>::insert(std::pair<Datacratic::Id, Datacratic::Id> const&, RTBKIT::FinishedInfo const&, Datacratic::Date) + 0x52d in build/x86_64/bin/libpost_auction.so + 0x5bd1dbuild/x86_64/bin/libpost_auction.so(_ZN10Datacratic10TimeoutMapISt4pairINS_2IdES2_EN6RTBKIT12FinishedInfoEE6insertERKS3_RKS5_NS_4DateE+0x52d) [0x7fd43f204d1d]
03: 0x0x7fd43f1e3483 at RTBKIT::PostAuctionLoop::doBidResult(Datacratic::Id const&, Datacratic::Id const&, RTBKIT::SubmissionInfo const&, RTBKIT::Amount, Datacratic::Date, RTBKIT::BidStatus, std::string const&, std::string const&, RTBKIT::UserIds const&) + 0xbb3 in build/x86_64/bin/libpost_auction.so + 0x3a483build/x86_64/bin/libpost_auction.so(_ZN6RTBKIT15PostAuctionLoop11doBidResultERKN10Datacratic2IdES4_RKNS_14SubmissionInfoENS_6AmountENS1_4DateENS_9BidStatusERKSsSC_RKNS_7UserIdsE+0xbb3) [0x7fd43f1e3483]
04: 0x0x7fd43f1e458b at RTBKIT::PostAuctionLoop::checkExpiredAuctions() + 0x1cb in build/x86_64/bin/libpost_auction.so + 0x3b58bbuild/x86_64/bin/libpost_auction.so(_ZN6RTBKIT15PostAuctionLoop20checkExpiredAuctionsEv+0x1cb) [0x7fd43f1e458b]
05: 0x0x7fd43ef5b593 at Datacratic::PeriodicEventSource::processOne() + 0xb3 in build/x86_64/bin/libservices.so + 0xa2593build/x86_64/bin/libservices.so(_ZN10Datacratic19PeriodicEventSource10processOneEv+0xb3) [0x7fd43ef5b593]
06: 0x0x7fd43ef456dd at Datacratic::MessageLoop::processOne() + 0x11d in build/x86_64/bin/libservices.so + 0x8c6ddbuild/x86_64/bin/libservices.so(_ZN10Datacratic11MessageLoop10processOneEv+0x11d) [0x7fd43ef456dd]
07: 0x0x7fd43ef43be9 at Datacratic::MessageLoop::runWorkerThread() + 0xc9 in build/x86_64/bin/libservices.so + 0x8abe9build/x86_64/bin/libservices.so(_ZN10Datacratic11MessageLoop15runWorkerThreadEv+0xc9) [0x7fd43ef43be9]
08: 0x0x7fd43ef44508 in build/x86_64/bin/libservices.so + 0x8b508build/x86_64/bin/libservices.so(+0x8b508) [0x7fd43ef44508]
09: 0x0x7fd43dc74c19 in /home/rtbkit/local/lib/libboost_thread.so.1.47.0 + 0xdc19/home/rtbkit/local/lib/libboost_thread.so.1.47.0(+0xdc19) [0x7fd43dc74c19]
10: 0x0x7fd43f82fefc in /lib/x86_64-linux-gnu/libpthread.so.0 + 0x7efc/lib/x86_64-linux-gnu/libpthread.so.0(+0x7efc) [0x7fd43f82fefc]
11: 0x0x7fd43df66f8d at clone + 0x6d in /lib/x86_64-linux-gnu/libc.so.6 + 0xe4f8d/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd43df66f8d]
How to reproduce:
Sometimes you need to repeat steps 2-4
I am getting following message while i run the demo stack launcher:
launch fixed-price-agent
process 30793 killed by signal 6
crash! router detected at Thu May 16 06:22:19 2013
launch router
process 30813 killed by signal 6
process 30803 killed by signal 6
crash! post-auction detected at Thu May 16 06:22:24 2013
launch post-auction
crash! monitor detected at Thu May 16 06:22:29 2013
launch monitor
process 30819 killed by signal 6
crash! augmentor detected at Thu May 16 06:22:34 2013
launch augmentor
process 30827 killed by signal 6
crash! mock-ad-server detected at Thu May 16 06:22:40 2013
launch mock-ad-server
process 30839 killed by signal 6
crash! logger detected at Thu May 16 06:22:52 2013
launch logger
process 30851 killed by signal 6
process 30860 killed by signal 6
crash! banker detected at Thu May 16 06:22:57 2013
launch banker
crash! agent-configuration detected at Thu May 16 06:23:02 2013
launch agent-configuration
process 30869 killed by signal 6
crash! fixed-price-agent detected at Thu May 16 06:23:09 2013
launch fixed-price-agent
process 30887 killed by signal 6
crash! router detected at Thu May 16 06:23:32 2013
launch router
process 30901 killed by signal 6
process 30896 killed by signal 6
crash! monitor detected at Thu May 16 06:23:37 2013
launch monitor
And no key combination is working e.,g CTRL+B and 7 gives the following error:
d): server refused to accept the client
2013-05-16 06:24:52,615:31065(0x7f29ee07d700):ZOO_ERROR@handle_socket_error_msg@1697: Socket [127.0.0.1:2181] zk retcode=-4, errno=111(Connection refused): server refused to accept the client
2013-05-16 06:24:52,949:31065(0x7f29ee07d700):ZOO_ERROR@handle_socket_error_msg@1697: Socket [127.0.0.1:2181] zk retcode=-4, errno=111(Connection refused): server refused to accept the client
2013-05-16 06:24:53,283:31065(0x7f29ee07d700):ZOO_ERROR@handle_socket_error_msg@1697: Socket [127.0.0.1:2181] zk retcode=-4, errno=111(Connection refused): server refused to accept the client
2013-05-16 06:24:53,617:31065(0x7f29ee07d700):ZOO_ERROR@handle_socket_error_msg@1697: Socket [127.0.0.1:2181] zk retcode=-4, errno=111(Connection refused): server refused to accept the client
I am getting the following issue:
make: Target `all' not remade because of errors.
in the same way if i proceed by leaving this issue, i got stuck here:
"
make nodejs_dependencies"
npm install .
/bin/bash: npm: command not found
make: *** [nodejs_dependencies] Error 127
Kindly help in this Regard
Murtaza
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.