mozilla-services / lua_sandbox Goto Github PK

Generic Lua sandbox for dynamic data analysis

License: Other

CMake 1.58% C 92.81% Lua 4.94% Shell 0.67%

lua_sandbox's Introduction

Lua Sandbox Library

Overview

Sandboxes provide a dynamic and isolated execution environment for data parsing, transformation, and analysis. They allow access to data without jeopardizing the integrity or performance of the processing infrastructure. This broadens the audience that the data can be exposed to and facilitates new uses of the data (i.e. debugging, monitoring, dynamic provisioning, SLA analysis, intrusion detection, ad-hoc reporting, etc.)

The Lua sandbox is a library allowing customized control over the Lua execution environment including functionality like global data preservation/restoration on shutdown/startup, output collection in textual or binary formats and an array of parsers for various data types (Nginx, Apache, Syslog, MySQL and many RFC grammars)

These libraries and utilities have been mostly extracted from Hindsight. The goal was to decouple the Heka/Hindsight functionality from any particular infrastructure and make it embeddable into any tool or language.

Features

small - memory requirements are as little as 8 KiB for a basic sandbox
fast - microsecond execution times
stateful - ability to resume where it left off after a restart/reboot
isolated - failures are contained and malfunctioning sandboxes are terminated. Containment is defined in terms of restriction to the operating system, file system, libraries, memory use, Lua instruction use, and output size.

Full Documentation

Installation

Prerequisites

C compiler (GCC 4.7+, Visual Studio 2013)
CMake (3.0+) - http://cmake.org/cmake/resources/software.html
Git http://git-scm.com/download

Optional (used for documentation)

Graphviz (2.28.0) - http://graphviz.org/Download..php
Doxygen (1.8.11+) - http://www.stack.nl/~dimitri/doxygen/download.html#latestsrc
gitbook (2.3) - https://www.gitbook.com/
lua (5.1) - https://www.lua.org/download.html

CMake Build Instructions

git clone https://github.com/mozilla-services/lua_sandbox.git
cd lua_sandbox
mkdir release
cd release

# UNIX
cmake -DCMAKE_BUILD_TYPE=release ..
make

# Windows Visual Studio 2013
cmake -DCMAKE_BUILD_TYPE=release -G "NMake Makefiles" ..
nmake

ctest
cpack -G TGZ # (DEB|RPM|ZIP)

Releases

The main branch is the current release and is considered stable at all times.
New versions can be released as frequently as every two weeks (our sprint cycle). The only exception would be for a high priority patch.
All active work is flagged with the sprint milestone and tracked in the project dashboard.
New releases occur the day after the sprint finishes.
- The version in the dev branch is updated
- The changes are merged into main
- A new tag is created

Contributions

All pull requests must be made against the dev branch, direct commits to main are not permitted.
All non trivial contributions should start with an issue being filed (if it is a new feature please propose your design/approach before doing any work as not all feature requests are accepted).

lua_sandbox's People

Contributors

Stargazers

Watchers

Forkers

cryptolab nixm0nk3y davidbirdsong liexusong michaelgibson boostrack-oss asenchi whd hynd h5eye cluo sathieu jedisct1 carlanton bdutta19 magneticcoffee spolakh vlastv helioslite zeusf1 timurb xrl mattltw skade tidal-loop dunderjester mattmoyer nathwill tianlin bbinet ifzz yan771012 robison mirwan nickchappell chruss2 acmiyaguchi yuanfeng0905 mei-rune crispin-linden vicglarson washort miffa brolerhack jbuck composit yoojinl ameihm0912 lyft tclh123 alexandruavadanii yaxc z1617 learnforpractice 657794040 mozilla-github-standards happileming ugiwgh moneytech zhaoshijian nikolas 10allday-services trink ishigoemon dynamic2500 litew dwdcth

lua_sandbox's Issues

lsb_serialize_double needs two forms

one for output and the other for serialization due the nan and inf handling

syslog decoder unable to parse structured data

RFC5424 includes support structured data, that modules/syslog.lua is trying to parse

Structured data looks like:

[abcd@42 software="test" swVersion="0.0.1"]

The parser for this is:

local hostname        = nilvalue + printusascii^-255
local sd_name         = printusascii^-32
local param_name      = sd_name
local param_value     = utf8_string
local sd_param        = param_name * '="' * param_value * '"'
local sd_id           = sd_name
local sd_element      = l.P"[" * sd_id * (sp * sd_param)^0 * "]"

Unfortunately, this grammar is incorrect, and the parser is actually unable to parse sd_element.

printusascii includes the = character, sd_name is greedier than expected. RFC 5424 explicitly prevents this in its definition, by disallowing a few characters:

SD-NAME         = 1*32PRINTUSASCII ; except '=', SP, ']', %d34 (")

Changing the Lua equivalent to:

local sd_name         = (printusascii - l.S'=]"' - sp)^-32

Makes the parser properly handle property names.

Unfortunately, param_value has the same issue, utf8_string includes the " character.

A terrible workaround is:

local param_value     = (octet - l.P'"')^0

But of course, this would miserably fail with values containing escaped characters.

The recently added strict module requires debug.getinfo

Either we need to add debug module support or remove strict.lua

My vote is for removing strict.lua and leaving the entire debug module out of the sandbox.

unit tests fail for datetime due to year change

$ ctest
..........
6: Test command: /Users/timurb/github/lua_sandbox/release/src/test/test_lua_sandbox
6: Environment variables:
6:  DYLD_LIBRARY_PATH=../lib
6: Test timeout computed to be: 1500
6: cur_mem 13994
6: max_mem 19177
6: max_ins 7
6: max_out 0
6: line: 848 (result == 0) process() received: 1 process() lua/lpeg_date_time.lua:63: test: 1 Feb 10 16:46:36 expected: 1.42359e+18 received: 1.45512e+18
6: Tests run: 17
6/6 Test #6: test_sandbox .....................***Failed    0.31 sec

This happens because the timestamp is expected for year 2015 and now the 2016 has come.

If I place 1455122796000000000 to https://github.com/mozilla-services/lua_sandbox/blob/master/src/test/lua/lpeg_date_time.lua#L62 the tests starts to pass fine but then a similar case appears in a different place:

6: line: 848 (result == 0) process() received: 1 process() lua/lpeg_syslog.lua:12: 1.455027421e+18

Looking at https://github.com/mozilla-services/lua_sandbox/blob/master/modules/date_time.lua#L250 it seems that the currently implemented function always returns current year and needs some patching to allow different year to be used.

Support hi-res timestamps in heka_encode_message_table

Currently if a Timestamp is not specified it is filled with a one second resolution time stamp.

nginx 'http_' variable consumes too few bytes

try this snippet

local clf = require 'common_log_format'
local grammar = clf.build_nginx_grammar('$http_accept_encoding')
local m = grammar:match('deflate, gzip')
print(m.http_accept_encoding)

I tried to patch, but I couldn't figure out how or where the 'http_*' grammar is defined.

Correct the syslogtag grammar

mozilla-services/heka#771

Investigate high but stable memory usage in common_log_format.lua

mozilla-services/heka#758

Tests fail to load libuasb on OSX

Builds OK, but tests fail with:

  dyld: Library not loaded: libluasb.0.dylib
  Referenced from: /Users/vsterzhanov/lua_sandbox/release/src/test/test_lua_sandbox
  Reason: image not found

Output of otool -L src/test/test_lua_sandbox:

src/test/test_lua_sandbox:
    /Users/vsterzhanov/lua_sandbox/release/src/libluasandbox.0.dylib (compatibility version 0.0.0,  current version 0.8.0)
    libluasb.0.dylib (compatibility version 0.0.0, current version 5.1.5)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)

lsb_heka_cat message matcher does not handle escaped char

"Uuid < '\008' is treated as a literal string instead of char 8. In the configuration files the escaping is handled by the Lua parser

Protobuf encoding - value_type of BYTES is not being properly set

If a representation is not set the value_type is defaulted to string
When the byte representation is added it should come after value_type

decode_message failures should just throw an error instead of returning nil and an error message

This will make it more consistent with inject_message or cjson.decode.

Investigate replacing the bloom filter with a cuckoo filter

"Cuckoo filters support adding and removing items dynamically while achieving even higher performance than Bloom filters."

http://www.eecs.harvard.edu/~michaelm/postscripts/cuckoo-conext2014.pdf

Tearing down blocked inputs can segfault

If an input plugin is blocked on a system call and is orphaned and then becomes unblocked during the process shutdown there is a race condition where it can segfault while exiting.

the heka stream reader needed bytes calculation is incorrect

This occurs when the scanpos is not at the beginning and a message length/header has been located

Add heka protobuf decoding into the sandbox

Error when running make install

I get an error when running make install:

$ cd release
$ make  # completes successfully
[…]
$ make install
[…]
-- Installing: /usr/local/bin/lsb_heka_cat
-- Removed runtime path from "/usr/local/bin/lsb_heka_cat"
CMake Error at src/cli/cmake_install.cmake:54 (file):
  file INSTALL cannot find "/lua_sandbox/src/cli/scripts".
Call Stack (most recent call first):
  src/cmake_install.cmake:89 (include)
  cmake_install.cmake:56 (include)

Makefile:66: recipe for target 'install' failed
make: *** [install] Error 1

Is this a known issue? Thanks.

Remove the set_header 'avg' aggregation method

The values in the circular buffer should always represent the raw values i.e. sum and count to make aggregation possible. The original use, of this method, was for presentation labels on the Heka dashboard when displaying pre calculated averages.

lua_sandbox fails to build on win7(32bit)

two issues:
1). when it comes to build lua_sandbox under win7(32bit):
CMake Error at src/CMakeLists.txt:35 (install):
install DIRECTORY does not allow "COMPONENT" after PATTERN or REGEX.
2). I rewrite src/CMakeLists.txt:35 to

install(DIRECTORY "${EP_BASE}/bin/" DESTINATION lib COMPONENT core PATTERN "*.dll" )
and rebuild again:

[ 22%] Performing build step for 'lua_bloom_filter'
Scanning dependencies of target bloom_filter
Building C object CMakeFiles/bloom_filter.dir/lua_bloom_filter.c.obj
Linking C shared module bloom_filter.dll
C:\yottabyte_heka\build\ep_base\Build\lua_sandbox\ep_base\lib\libluasb.dll.a: er
ror adding symbols: Archive has no index; run ranlib to add one
collect2.exe: error: ld returned 1 exit status
CMakeFiles\bloom_filter.dir\build.make:115: recipe for target 'bloom_filter.dll'
failed
mingw32-make[8]: *** [bloom_filter.dll] Error 1
CMakeFiles\Makefile2:927: recipe for target 'CMakeFiles/bloom_filter.dir/all' fa
iled
mingw32-make[7]: *** [CMakeFiles/bloom_filter.dir/all] Error 2
Makefile:142: recipe for target 'all' failed
mingw32-make[6]: *** [all] Error 2
CMakeFiles\lua_bloom_filter.dir\build.make:108: recipe for target 'ep_base/Stamp
/lua_bloom_filter/lua_bloom_filter-build' failed
mingw32-make[5]: *** [ep_base/Stamp/lua_bloom_filter/lua_bloom_filter-build] Err
or 2
CMakeFiles\Makefile2:962: recipe for target 'CMakeFiles/lua_bloom_filter.dir/all
' failed
mingw32-make[4]: *** [CMakeFiles/lua_bloom_filter.dir/all] Error 2
Makefile:142: recipe for target 'all' failed
mingw32-make[3]: *** [all] Error 2

Improve time series (cbuf) data analysis

The mww and roc algorithms have been useful in identifying anomalies in the graph data. However, in FxA and particularly Sync, these anomalies occur very frequently leading to MANY inactionable alerts. The causes range from known bugs to recurring traffic patterns which the current analysis cannot account for.

Cannot compile on ArchLinux

Linux 4.6.2-1-ARCH
gcc (GCC) 6.1.1 20160602

Compile failed

/tmp/lua_sandbox/src/luasandbox_modules.c:64:30: error: array type has incomplete element type ‘struct luaL_reg’
 static const struct luaL_reg hashlib_f[] =
                              ^~~~~~~~~
/tmp/lua_sandbox/src/luasandbox_modules.c: In function ‘luaopen_lsb_hash’:
/tmp/lua_sandbox/src/luasandbox_modules.c:75:3: error: implicit declaration of function ‘luaL_register’ [-Werror=implicit-
function-declaration]

Change the sandbox size to size_t

Currently the size is stored in an unsigned int but after removing the size restriction this can now be overflowed.

Add a preprocessor to manage constants

Pull in common constants like
SEC_IN_HOUR
SEC_IN_DAY
SEC_IN_WEEK

and possibly common functions like get_day(ns) to replace the many instances of

local day = floor(ts / (SEC_IN_DAY * 1e9))

I am still undecided on how useful this may actually be since in most cases it will pull in a bunch of stuff that is not needed and slightly increase access times for the variables.

Update the LuaJIT build with the new module handling

Remove the sandbox limit caps

The current limits are hardcoded to the Heka maximums.
memory: 8MiB
instructions: 1M
output: 63KiB

There should be no limits. If someone wants to use the sandbox with no limitations the values should be set to zero, otherwise the user should be able to specify whatever they want.

How to require a lua moudle in a sandbox?

Hi, all

I write a lua sandbox

require "ltn12"
  ...

and when started hekad I got this error

2016/01/22 13:25:37 Error making runner for log4j_filter: Initialization failed for 'log4j_filter': Init() /usr/share/heka/nxin/log4j_filter.lua:3: module 'ltn12' not found:
        no file '/usr/share/heka/lua_modules/ltn12.lua'
        no file '/usr/share/heka/lua_modules/ltn12.so'

but require "cjson" works fine

I find this moudle is in path lua_io_modules

# find /usr/share/heka/ -name ltn12.*
/usr/share/heka/lua_io_modules/ltn12.lua

I tried require "lua_io_modules.ltn12" and still not work
how can I require this muodle in a sandbox ?

Fix the race condition in os.date()

Lua 5.1 uses localtime/gmtime which is not thread safe and can produce erroneous results when used from multiple sandboxes. Patch the Lua code to use localtime_r/gmtime_r if they are available (like in Lua 5.2)

Parse android and ios Firefox Sync user agents

It would be awesome if normalize_user_agent could parse the user agent strings sent by the mobile sync clients.

https://github.com/mozilla/gecko-dev/blob/233f30f2377f3df0f3388721901681f432b813fb/mobile/android/services/src/main/java/org/mozilla/gecko/sync/SyncConstants.java#L17

https://github.com/mozilla/firefox-ios/blob/5238e873e77e9ad3e699f926d12f61ccafabdc11/Shared/UserAgent.swift#L11

Create a sandbox package

rpm, deb, and zip

See: mozilla-services/heka#1329

Ops is seeing duplicate UUID's being generated from multiple sandboxes

There are a few issues:

between separate processes (they most likely used the same seed, since it is just time_t). A better seed should be used.
srand/rand are not thread safe in C so the behaviour probably isn't matching expectations
2a) the Lua math library uses srand/rand so it should also be addressed there.
the modulo operation does not generate uniformly distributed random numbers in the span (lower numbers are slightly more likely).

IE 11 user agent in nginx access logs

Hi,

It seems like the standard heka nginx access decoder doesn't recognize the IE11 user agent. I don't see the useragent in common_log_format.lua.

This is what a IE11 useragent looks like:

Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko

This is how the message looks in Elasticsearch:

t_index         nginx-2015.12.09
t_type      nginx
#body_bytes_sent        52
tcookie_JSESSIONID      5e4a6a6af748d8a628d806855c7d
thttp_referer       https://portal.website.com/path/
thttp_user_agent        Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko
tremote_addr        165.x.x.x
trequest        POST /path/main.jsf HTTP/1.1
#request_time       0.01
#status         200
tuser_agent_os      Windows 7

Consider adding lua-MessagePack module

I was wondering if it would be possible to add https://github.com/fperrad/lua-MessagePack/ as an available module. This would give a path to develop Heka encoders/decoders that communicate with http://www.fluentd.org/ natively and are maybe able to replace it.

Compilation fails on FreeBSD

Heka build for the recent git pull fails:

$ source build.sh
-- Configuring done
-- Generating done
-- Build files have been written to: /home/macbre/src/heka/build
[  4%] Built target gomock
[  8%] Built target go-simplejson
[ 13%] Built target whisper-go
[ 18%] Built target go-notify
[ 23%] Built target toml
[ 27%] Built target amqp
[ 32%] Built target raw
[ 36%] Built target slices
[ 41%] Built target sets
[ 46%] Built target goamz
[ 51%] Built target gospec
[ 56%] Built target g2s
[ 60%] Built target heka-docs
[ 65%] Built target xmlpath
[ 70%] Built target go-uuid
[ 75%] Built target goprotobuf
[ 80%] Built target heka-mozsvc-plugins
[ 80%] Built target GoPackages
[ 81%] Performing update step (git fetch) for 'luasandbox-0_1_0'
HEAD is now at dcf16dd... Update circular_buffer.md
[ 82%] Performing configure step for 'luasandbox-0_1_0'
Not searching for unused variables given on the command line.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/macbre/src/heka/build/ep_base/Build/luasandbox-0_1_0
[ 82%] Performing build step for 'luasandbox-0_1_0'
[ 25%] Built target lua-5_1_5
[ 51%] Built target lpeg-0_12
[ 77%] Built target lua-cjson-2_1_0
[ 96%] Built target luasandbox
Linking C executable test_lua_sandbox
/usr/bin/ld: cannot find -ldl
*** [src/test/test_lua_sandbox] Error code 1

Stop in /usr/home/macbre/src/heka/build/ep_base/Build/luasandbox-0_1_0.
*** [src/test/CMakeFiles/test_lua_sandbox.dir/all] Error code 1

Stop in /usr/home/macbre/src/heka/build/ep_base/Build/luasandbox-0_1_0.
*** [all] Error code 1

Stop in /usr/home/macbre/src/heka/build/ep_base/Build/luasandbox-0_1_0.
*** [ep_base/Stamp/luasandbox-0_1_0/luasandbox-0_1_0-build] Error code 1

Stop in /usr/home/macbre/src/heka/build.
*** [CMakeFiles/luasandbox-0_1_0.dir/all] Error code 1

Stop in /usr/home/macbre/src/heka/build.
*** [all] Error code 1

Stop in /usr/home/macbre/src/heka/build.

$ uname  -or
FreeBSD 9.1-STABLE

-ldl flag should not be used on FreeBSD systems:

lua_sandbox/src/CMakeLists.txt

Line 43 in 53ba169

-ldl -lm

Create a standalone message match parser in the luasandboxutil lib

Remove the dependency on the sandbox and create a standalone match parser in the luasandboxutil lib https://github.com/mozilla-services/lua_sandbox/blob/master/include/luasandbox/heka/message_matcher.h#L32

The sandbox message matcher module created here: faadb41 is as temporary measure. The module should go away and be replaced by a simple create_message_matcher() function call (no builder wrapper required)

Properly handle the escaped quote characters in the common log format

log_format = 'request = "%r"'

Should properly parse:

request = "GET /blah/blah/plugins/special_chars/char_map.php?width=233</script>&height=233<script>alert(\"foo\")</script> HTTP/1.1"

Hmm, into the literal string or with all the escaping replace?

Pull the common host/hostname grammar into a common module

https://github.com/mozilla-services/lua_sandbox/pull/128/files#r60966006

No luasandbox/lua.h

After runnging cmake -DCMAKE_BUILD_TYPE=release .. and Makefile has been gernerated, I'm going to make, but came up with an error.
Here is the error message

[  1%] Building C object src/util/CMakeFiles/luasandboxutil.dir/heka_message.c.o
[  2%] Building C object src/util/CMakeFiles/luasandboxutil.dir/heka_message_matcher.c.o
In file included from /root/repo/lua_sandbox/src/util/heka_message_matcher.c:15:0:
/root/repo/lua_sandbox/release/ep_base/include/luasandbox.h:68:28: fatal error: luasandbox/lua.h: No such file or directory
#include "luasandbox/lua.h"

It needs patch when installed on CentOS 7

I think it'd better add it in Prerequisites.

Data restoration fails with 'constant table overflow'

The restoration file contained over INT_MAX-2 literals + real numbers. We should be a able to overcome the limit by breaking the restore into multiple scopes.

Apache Log Format Bug

%T should be parsed as a double, but is parsed as an int. This causes request times such as "0.23" to be parsed as "0" in the heka message.

I think "integer" in common_log_format.lua:163 should be changed to "double".

https://github.com/mozilla-services/lua_sandbox/blob/dev/modules/common_log_format.lua#L163

Fix lua path/cpath escaping in the windows configuration

Investigate splitting out some of the sandbox libraries into proper Lua rocks.

cbuf, bloom_filter, hyperloglog

lua_sandbox does not compile anymore on ARM raspberry pi device

It seems to failed during rapidjson compilation:

[ 96%] Built target test_util
Scanning dependencies of target luasandboxheka
[ 97%] Building C object src/heka/CMakeFiles/luasandboxheka.dir/message.c.o
[ 97%] Building CXX object src/heka/CMakeFiles/luasandboxheka.dir/rapidjson.cpp.o
In file included from /root/lua_sandbox/src/heka/rapidjson.cpp:17:0:
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/writer.h: In instantiation of 'bool rapidjson::Writer<OutputStream, SourceEncoding, TargetEncoding, StackAllocator, writeFlags>::WriteString(const Ch*, rapidjson::SizeType) [with OutputStream = OutputBufferWrapper; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator; unsigned int writeFlags = 0u; rapidjson::Writer<OutputStream, SourceEncoding, TargetEncoding, StackAllocator, writeFlags>::Ch = char; rapidjson::SizeType = unsigned int]':
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/writer.h:177:39:   required from 'bool rapidjson::Writer<OutputStream, SourceEncoding, TargetEncoding, StackAllocator, writeFlags>::String(const Ch*, rapidjson::SizeType, bool) [with OutputStream = OutputBufferWrapper; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator; unsigned int writeFlags = 0u; rapidjson::Writer<OutputStream, SourceEncoding, TargetEncoding, StackAllocator, writeFlags>::Ch = char; rapidjson::SizeType = unsigned int]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:1782:99:   required from 'bool rapidjson::GenericValue<Encoding, Allocator>::Accept(Handler&) const [with Handler = rapidjson::Writer<OutputBufferWrapper>; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>]'
/root/lua_sandbox/src/heka/rapidjson.cpp:611:19:   required from here
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/writer.h:371:18: error: comparison is always true due to limited range of data type [-Werror=type-limits]
In file included from /root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:20:0,
                 from /root/lua_sandbox/src/heka/rapidjson.cpp:11:
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h: In instantiation of 'rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Token rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Tokenize(rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Ch) [with SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Ch = char]':
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:1724:41:   required from 'rapidjson::ParseResult rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::IterativeParse(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::MemoryStream; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:477:58:   required from 'rapidjson::ParseResult rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Parse(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::MemoryStream; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2146:9:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 0u; SourceEncoding = rapidjson::UTF8<>; InputStream = rapidjson::MemoryStream; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/src/heka/rapidjson.cpp:549:48:   required from here
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:1382:9: error: comparison is always true due to limited range of data type [-Werror=type-limits]
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h: In instantiation of 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseStringToStream(InputStream&, OutputStream&) [with unsigned int parseFlags = 0u; SEncoding = rapidjson::UTF8<>; TEncoding = rapidjson::UTF8<>; InputStream = rapidjson::MemoryStream; OutputStream = rapidjson::MemoryStream; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]':
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:791:13:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseString(InputStream&, Handler&, bool) [with unsigned int parseFlags = 0u; InputStream = rapidjson::MemoryStream; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:1303:23:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseValue(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::MemoryStream; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:491:13:   required from 'rapidjson::ParseResult rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Parse(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::MemoryStream; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2146:9:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 0u; SourceEncoding = rapidjson::UTF8<>; InputStream = rapidjson::MemoryStream; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/src/heka/rapidjson.cpp:549:48:   required from here
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:836:17: error: comparison is always true due to limited range of data type [-Werror=type-limits]
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h: In instantiation of 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseStringToStream(InputStream&, OutputStream&) [with unsigned int parseFlags = 0u; SEncoding = rapidjson::UTF8<>; TEncoding = rapidjson::UTF8<>; InputStream = rapidjson::MemoryStream; OutputStream = rapidjson::GenericReader<rapidjson::UTF8<>, rapidjson::UTF8<> >::StackStream<char>; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]':
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:800:13:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseString(InputStream&, Handler&, bool) [with unsigned int parseFlags = 0u; InputStream = rapidjson::MemoryStream; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:1303:23:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseValue(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::MemoryStream; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:491:13:   required from 'rapidjson::ParseResult rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Parse(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::MemoryStream; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2146:9:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 0u; SourceEncoding = rapidjson::UTF8<>; InputStream = rapidjson::MemoryStream; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/src/heka/rapidjson.cpp:549:48:   required from here
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:836:17: error: comparison is always true due to limited range of data type [-Werror=type-limits]
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h: In instantiation of 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseStringToStream(InputStream&, OutputStream&) [with unsigned int parseFlags = 9u; SEncoding = rapidjson::UTF8<>; TEncoding = rapidjson::UTF8<>; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; OutputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]':
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:791:13:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseString(InputStream&, Handler&, bool) [with unsigned int parseFlags = 9u; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:1303:23:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseValue(InputStream&, Handler&) [with unsigned int parseFlags = 9u; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:491:13:   required from 'rapidjson::ParseResult rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Parse(InputStream&, Handler&) [with unsigned int parseFlags = 9u; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2146:9:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 9u; SourceEncoding = rapidjson::UTF8<>; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2162:65:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 9u; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2187:60:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseInsitu(rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Ch*) [with unsigned int parseFlags = 8u; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Ch = char]'
/root/lua_sandbox/src/heka/rapidjson.cpp:540:68:   required from here
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:836:17: error: comparison is always true due to limited range of data type [-Werror=type-limits]
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h: In instantiation of 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseStringToStream(InputStream&, OutputStream&) [with unsigned int parseFlags = 9u; SEncoding = rapidjson::UTF8<>; TEncoding = rapidjson::UTF8<>; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; OutputStream = rapidjson::GenericReader<rapidjson::UTF8<>, rapidjson::UTF8<> >::StackStream<char>; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]':
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:800:13:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseString(InputStream&, Handler&, bool) [with unsigned int parseFlags = 9u; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:1303:23:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseValue(InputStream&, Handler&) [with unsigned int parseFlags = 9u; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:491:13:   required from 'rapidjson::ParseResult rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Parse(InputStream&, Handler&) [with unsigned int parseFlags = 9u; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2146:9:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 9u; SourceEncoding = rapidjson::UTF8<>; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2162:65:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 9u; InputStream = rapidjson::GenericInsituStringStream<rapidjson::UTF8<> >; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2187:60:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseInsitu(rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Ch*) [with unsigned int parseFlags = 8u; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Ch = char]'
/root/lua_sandbox/src/heka/rapidjson.cpp:540:68:   required from here
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:836:17: error: comparison is always true due to limited range of data type [-Werror=type-limits]
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h: In instantiation of 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseStringToStream(InputStream&, OutputStream&) [with unsigned int parseFlags = 0u; SEncoding = rapidjson::UTF8<>; TEncoding = rapidjson::UTF8<>; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; OutputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]':
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:791:13:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseString(InputStream&, Handler&, bool) [with unsigned int parseFlags = 0u; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:1303:23:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseValue(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:491:13:   required from 'rapidjson::ParseResult rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Parse(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2146:9:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 0u; SourceEncoding = rapidjson::UTF8<>; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2211:57:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Parse(const typename SourceEncoding::Ch*) [with unsigned int parseFlags = 0u; SourceEncoding = rapidjson::UTF8<>; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >; typename SourceEncoding::Ch = char]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2220:47:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Parse(const Ch*) [with unsigned int parseFlags = 0u; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Ch = char]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2227:45:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Parse(const Ch*) [with Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Ch = char]'
/root/lua_sandbox/src/heka/rapidjson.cpp:118:23:   required from here
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:836:17: error: comparison is always true due to limited range of data type [-Werror=type-limits]
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h: In instantiation of 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseStringToStream(InputStream&, OutputStream&) [with unsigned int parseFlags = 0u; SEncoding = rapidjson::UTF8<>; TEncoding = rapidjson::UTF8<>; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; OutputStream = rapidjson::GenericReader<rapidjson::UTF8<>, rapidjson::UTF8<> >::StackStream<char>; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]':
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:800:13:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseString(InputStream&, Handler&, bool) [with unsigned int parseFlags = 0u; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:1303:23:   required from 'void rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::ParseValue(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:491:13:   required from 'rapidjson::ParseResult rapidjson::GenericReader<SourceEncoding, TargetEncoding, StackAllocator>::Parse(InputStream&, Handler&) [with unsigned int parseFlags = 0u; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; Handler = rapidjson::GenericDocument<rapidjson::UTF8<> >; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2146:9:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::ParseStream(InputStream&) [with unsigned int parseFlags = 0u; SourceEncoding = rapidjson::UTF8<>; InputStream = rapidjson::GenericStringStream<rapidjson::UTF8<> >; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2211:57:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Parse(const typename SourceEncoding::Ch*) [with unsigned int parseFlags = 0u; SourceEncoding = rapidjson::UTF8<>; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >; typename SourceEncoding::Ch = char]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2220:47:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Parse(const Ch*) [with unsigned int parseFlags = 0u; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Ch = char]'
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/document.h:2227:45:   required from 'rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>& rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Parse(const Ch*) [with Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; StackAllocator = rapidjson::CrtAllocator; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator> = rapidjson::GenericDocument<rapidjson::UTF8<> >; rapidjson::GenericDocument<Encoding, Allocator, StackAllocator>::Ch = char]'
/root/lua_sandbox/src/heka/rapidjson.cpp:118:23:   required from here
/root/lua_sandbox/release/ep_base/Source/rapidjson/include/rapidjson/reader.h:836:17: error: comparison is always true due to limited range of data type [-Werror=type-limits]
cc1plus: all warnings being treated as errors
src/heka/CMakeFiles/luasandboxheka.dir/build.make:77: recipe for target 'src/heka/CMakeFiles/luasandboxheka.dir/rapidjson.cpp.o' failed
make[2]: *** [src/heka/CMakeFiles/luasandboxheka.dir/rapidjson.cpp.o] Error 1
CMakeFiles/Makefile2:1841: recipe for target 'src/heka/CMakeFiles/luasandboxheka.dir/all' failed
make[1]: *** [src/heka/CMakeFiles/luasandboxheka.dir/all] Error 2
Makefile:136: recipe for target 'all' failed
make: *** [all] Error 2

Should I rather report this issue to the rapidjson repository instead?

lua_sandbox requires patched dependencies

Lua_sandbox currently requires:

a patched lua 5.1: LuaDist/lua@lua-5.1...trink:lua-5.1.5-sandbox
a patched lua_cjson: mpx/lua-cjson@master...trink:heka
(and maybe more)

This currently blocks the creation of a proper Debian package.

Add support for decode_message to also return hash based message fields

Currently the decode_message function returns a message with "Array Based Message Fields" as described in https://github.com/mozilla-services/lua_sandbox/blob/master/docs/heka/message.md#array-based-message-fields

It would be great if decode_message could also return a message with "Hash Based Message Fields" as described in https://github.com/mozilla-services/lua_sandbox/blob/master/docs/heka/message.md#hash-based-message-fields
Most of the time, fields names are unique and the hash based format is then easier to use in the sandbox than the array based format.

We could for example add an optional boolean parameter to the decode_message function to decide on which fields format to use (the default value would keep the current array based format).

What do you think?

some Firefox OS user agents not parsed correctly

@trink : We noticed a bunch of Firefox 18's (with no user_agent_os) in the logs (coming from Marketplace). @whd pulled up the user agent string:
Mozilla/5.0 (Mobile; ALCATELOneTouch4012A; rv:18.1) Gecko/18.1 Firefox/18.1.
Looks like the parser is missing this: https://github.com/mozilla-services/lua_sandbox/blob/dev/modules/common_log_format.lua#L314-331

fwiw, here's one of the dashboards where they show up: https://kibana.fxa.us-west-2.prod.mozaws.net/index.html#/dashboard/file/oauth_http_status.json

Add a context for the logger callback [breaking change]

This will allow log output to be bound to a specific context like an instance of the web based plugin tester.

compilation failing

i am try to compile lua_sandbox from master branch on mac osx and it's throwing following error during make.

Scanning dependencies of target copy_includes
[ 0%] Built target copy_includes
Scanning dependencies of target luasandboxutil
[ 0%] Building C object src/util/CMakeFiles/luasandboxutil.dir/heka_message.c.o
[ 1%] Building C object src/util/CMakeFiles/luasandboxutil.dir/heka_message_matcher.c.o
[ 2%] Building C object src/util/CMakeFiles/luasandboxutil.dir/heka_message_matcher_parser.c.o
[ 2%] Building C object src/util/CMakeFiles/luasandboxutil.dir/input_buffer.c.o
[ 3%] Building C object src/util/CMakeFiles/luasandboxutil.dir/output_buffer.c.o
[ 4%] Building C object src/util/CMakeFiles/luasandboxutil.dir/protobuf.c.o
[ 5%] Building C object src/util/CMakeFiles/luasandboxutil.dir/running_stats.c.o
[ 5%] Building C object src/util/CMakeFiles/luasandboxutil.dir/string.c.o
[ 6%] Building C object src/util/CMakeFiles/luasandboxutil.dir/string_matcher.c.o
[ 7%] Building C object src/util/CMakeFiles/luasandboxutil.dir/util.c.o
/Users/madhu/lua_sandbox/src/util/util.c:57:29: error: expected ')'
FILE fh = fopen(fn, "rb" CLOSE_ON_EXEC);
^
/Users/madhu/lua_sandbox/src/util/util.c:57:19: note: to match this '('
FILE *fh = fopen(fn, "rb" CLOSE_ON_EXEC);
^
1 error generated.
make[2]: ** [src/util/CMakeFiles/luasandboxutil.dir/util.c.o] Error 1
make[1]: *** [src/util/CMakeFiles/luasandboxutil.dir/all] Error 2
make: *** [all] Error 2

heka stream reader can integer overflow when reading the header length out of the buffer

The buffer changed from an unsigned char* to a char* a cast is now required

Allow lsb_output_protobuf to handle user data containing a sandbox output function

local fx_adu = hyperloglog.new()
local cbuf = circular_buffer.new(10, 1, 60, true)
local msg = {
Fields = {
fx_adu = {value = fx_adu, representation = "hll"},
cbufd = {value = cbuf.format("cbufd"), representation = "cbufd"},
}
}

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.