<appender name="KafkaStream" class="net.johnpage.kafka.KafkaLog4JAppender">
<param name="Topic" value="pega-logs"/>
<param name="kafkaProducerProperties" value="bootstrap.servers=127.0.0.1:9092 value.serializer=org.apache.kafka.common.serialization.StringSerializer key.serializer=org.apache.kafka.common.serialization.StringSerializer compression.type=snappy client.id=pega-pe-722"/>
<layout class="com.pega.pegarules.priv.LogLayoutJSON">
<param name="userFields" value="sourceLabel:pega-live,ls_type:pega-rules,src-vm:${hostName},src-node:nodename,src-env:${sys:pega.appservername}"/>
</layout>
</appender>This is a Pega Log4J v1 Appender integrated with a Kafka Producer. It streams events as they occur to a remote Kafka queue.
input {
kafka {
bootstrap_servers => "127.0.0.1:9092"
topics => ["pega-logs"]
codec => json
}
}
output {
stdout { codec => rubydebug }
}
Example of result events:
{
"@logtimestamp" => "2019-07-31T17:17:16.304Z",
"class" => "?",
"pegathread" => "STANDARD",
"source_host" => "srv001",
"file" => "?",
"src-node" => "nodename",
"@timestamp" => 2019-07-31T17:17:16.304Z,
"logger_name" => "com.pega.pegarules.search.internal.PRSearchProviderImpl",
"src-vm" => "",
"method" => "?",
"ls_type" => "pega-rules",
"@version" => 1,
"line_number" => "?",
"src-env" => "",
"sourceLabel" => "pega-live",
"level" => "INFO",
"message" => "Initialized full text search functionality for this node. Full-text search enabled for node F3962772FF1FEECF62815A313B27FC74",
"thread_name" => "StartupTaskUtil INITIALIZE_SEARCH"
}mvn clean install
- Download the kafka-pega-log4j-appender-1.1.jar and import in pega using Application->Distribution->Import wizard.
- Edit the prlogging.xml configuration file.
- Restart pega application server.
A typical Kafka Producer properties can be specified via file that might read as follows:
bootstrap.servers=a.domain.com:9092
value.serializer=org.apache.kafka.common.serialization.StringSerializer
key.serializer=org.apache.kafka.common.serialization.StringSerializer
security.protocol=SSL
ssl.truststore.location=a.kafka.client.truststore.jks
ssl.truststore.password=apasswordA complete reference to the producer properties is here.
Tested with Kafka 0.10. Should be backwards compatible with 0.90 and 0.82. These 3 versions rely on the following initialization of the Producer:
new KafkaProducer(Properties properties) To use a different version of Kafka, include the desired version on the classpath. Version-appropriate properties will need to be used.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent