Code Monkey home page Code Monkey logo

zipkin's Introduction

zipkin

Gitter chat Build Status Maven Central

Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in service architectures. Features include both the collection and lookup of this data.

If you have a trace ID in a log file, you can jump directly to it. Otherwise, you can query based on attributes such as service, operation name, tags and duration. Some interesting data will be summarized for you, such as the percentage of time spent in a service, and whether operations failed.

Trace view screenshot

The Zipkin UI also presents a dependency diagram showing how many traced requests went through each application. This can be helpful for identifying aggregate behavior including error paths or calls to deprecated services.

Dependency graph screenshot

Application’s need to be “instrumented” to report trace data to Zipkin. This usually means configuration of a tracer or instrumentation library. The most popular ways to report data to Zipkin are via http or Kafka, though many other options exist, such as Apache ActiveMQ, gRPC and RabbitMQ. The data served to the UI is stored in-memory, or persistently with a supported backend such as Apache Cassandra or Elasticsearch.

Quick-start

The quickest way to get started is to fetch the latest released server as a self-contained executable jar. Note that the Zipkin server requires minimum JRE 17+. For example:

curl -sSL https://zipkin.io/quickstart.sh | bash -s
java -jar zipkin.jar

You can also start Zipkin via Docker.

# Note: this is mirrored as ghcr.io/openzipkin/zipkin
docker run -d -p 9411:9411 openzipkin/zipkin

Once the server is running, you can view traces with the Zipkin UI at http://localhost:9411/zipkin.

If your applications aren't sending traces, yet, configure them with Zipkin instrumentation or try one of our examples.

Check out the zipkin-server documentation for configuration details, or Docker examples for how to use docker-compose.

Zipkin Slim

The slim build of Zipkin is smaller and starts faster. It supports in-memory and Elasticsearch storage, but doesn't support messaging transports like Kafka or RabbitMQ. If these constraints match your needs, you can try slim like below:

Running via Java:

curl -sSL https://zipkin.io/quickstart.sh | bash -s io.zipkin:zipkin-server:LATEST:slim zipkin.jar
java -jar zipkin.jar

Running via Docker:

# Note: this is mirrored as ghcr.io/openzipkin/zipkin-slim
docker run -d -p 9411:9411 openzipkin/zipkin-slim

Running via Homebrew:

brew install zipkin
# to run in foreground
zipkin
# to run in background
brew services start zipkin

Core Library

The core library is used by both Zipkin instrumentation and the Zipkin server.

This includes built-in codec for Zipkin's v1 and v2 json formats. A direct dependency on gson (json library) is avoided by minifying and repackaging classes used. The result is a 155k jar which won't conflict with any library you use.

Ex.

// All data are recorded against the same endpoint, associated with your service graph
localEndpoint = Endpoint.newBuilder().serviceName("tweetie").ip("192.168.0.1").build()
span = Span.newBuilder()
    .traceId("d3d200866a77cc59")
    .id("d3d200866a77cc59")
    .name("targz")
    .localEndpoint(localEndpoint)
    .timestamp(epochMicros())
    .duration(durationInMicros)
    .putTag("compression.level", "9");

// Now, you can encode it as json
bytes = SpanBytesEncoder.JSON_V2.encode(span);

Note: The above is just an example, most likely you'll want to use an existing tracing library like Brave

Core Library Requires Java 8+

The minimum Java language level of the core library is 8. This helps support those writing agent instrumentation. Version 2.x was the last to support Java 6.

Note: zipkin-reporter-brave does not use this library. So, brave still supports Java 6.

Storage Component

Zipkin includes a StorageComponent, used to store and query spans and dependency links. This is used by the server and those making collectors, or span reporters. For this reason, storage components have minimal dependencies, though require Java 17+.

Ex.

// this won't create network connections
storage = ElasticsearchStorage.newBuilder()
                              .hosts(asList("http://myelastic:9200")).build();

// prepare a call
traceCall = storage.spanStore().getTrace("d3d200866a77cc59");

// execute it synchronously or asynchronously
trace = traceCall.execute();

// clean up any sessions, etc
storage.close();

In-Memory

The InMemoryStorage component is packaged in zipkin's core library. It is neither persistent, nor viable for realistic work loads. Its purpose is for testing, for example starting a server on your laptop without any database needed.

Cassandra

The Cassandra component uses Cassandra 3.11.3+ features, but is tested against the latest patch of Cassandra 4.1.

This is the second generation of our Cassandra schema. It stores spans using UDTs, such that they appear like Zipkin v2 json in cqlsh. It is designed for scale, and uses a combination of SASI and manually implemented indexes to make querying larger data more performant.

Note: This store requires a job to aggregate dependency links.

Elasticsearch

The Elasticsearch component uses Elasticsearch 5+ features, but is tested against Elasticsearch 7-8.x and OpenSearch 2.x.

It stores spans as Zipkin v2 json so that integration with other tools is straightforward. To help with scale, this uses a combination of custom and manually implemented indexing.

Note: This store requires a spark job to aggregate dependency links.

Disabling search

The following API endpoints provide search features, and are enabled by default. Search primarily allows the trace list screen of the UI operate.

  • GET /services - Distinct Span.localServiceName
  • GET /remoteServices?serviceName=X - Distinct Span.remoteServiceName by Span.localServiceName
  • GET /spans?serviceName=X - Distinct Span.name by Span.localServiceName
  • GET /autocompleteKeys - Distinct keys of Span.tags subject to configurable whitelist
  • GET /autocompleteValues?key=X - Distinct values of Span.tags by key
  • GET /traces - Traces matching a query possibly including the above criteria

When search is disabled, traces can only be retrieved by ID (GET /trace/{traceId}). Disabling search is only viable when there is an alternative way to find trace IDs, such as logs. Disabling search can reduce storage costs or increase write throughput.

StorageComponent.Builder.searchEnabled(false) is implied when a zipkin is run with the env variable SEARCH_ENABLED=false.

Legacy (v1) components

The following components are no longer encouraged, but exist to help aid transition to supported ones. These are indicated as "v1" as they use data layouts based on Zipkin's V1 Thrift model, as opposed to the simpler v2 data model currently used.

MySQL

The MySQL v1 component uses MySQL 5.6+ features, but is tested against MariaDB 10.11.

The schema was designed to be easy to understand and get started with; it was not designed for performance. Ex spans fields are columns, so you can perform ad-hoc queries using SQL. However, this component has known performance issues: queries will eventually take seconds to return if you put a lot of data into it.

This store does not require a job to aggregate dependency links. However, running the job will improve performance of dependencies queries.

Running the server from source

The Zipkin server receives spans via HTTP POST and respond to queries from its UI. It can also run collectors, such as RabbitMQ or Kafka.

To run the server from the currently checked out source, enter the following. JDK 17+ is required to compile the source.

# Build the server and also make its dependencies
$ ./mvnw -q --batch-mode -DskipTests --also-make -pl zipkin-server clean install
# Run the server
$ java -jar ./zipkin-server/target/zipkin-server-*exec.jar

Artifacts

Server artifacts are under the maven group id io.zipkin Library artifacts are under the maven group id io.zipkin.zipkin2

Library Releases

Releases are at Sonatype and Maven Central

Library Snapshots

Snapshots are uploaded to Sonatype after commits to master.

Docker Images

Released versions of zipkin-server are published to Docker Hub as openzipkin/zipkin and GitHub Container Registry as ghcr.io/openzipkin/zipkin. See docker for details.

Helm Charts

Helm charts are available via helm repo add zipkin https://zipkin.io/zipkin-helm. See zipkin-helm for details.

Javadocs

https://zipkin.io/zipkin contains versioned folders with JavaDocs published on each (non-PR) build, as well as releases.

zipkin's People

Contributors

abesto avatar adriancole avatar anuraaga avatar cburroughs avatar codefromthecrypt avatar drolando avatar eirslett avatar fedj avatar igorwwwwwwwwwwwwwwwwwwww avatar jcchavezs avatar jeqo avatar joel-airspring avatar jonkerj avatar jorgheymans avatar llinder avatar logic-32 avatar making avatar michaelsembwever avatar minwoox avatar mrglaucus avatar mrproliu avatar oscerd avatar reta avatar sethp-jive avatar shakuzen avatar sprsquish avatar tacigar avatar timtebeek avatar virtuald avatar zeagord avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zipkin's Issues

Copy link to this search button

Imagine you've just searched for something and you want to share that results page. Similar to how Google maps has a link called something like "link to this map" we could have a "link to this search". It would create a link that recreates the search for another user.

Search broken on traces page?

Is it just me or is the search bar completely unusable? No queries are returning any results.

Also, I wonder if ctrl + F works for your guys? For me, In Firefox, ctrl + F doesn't highlight the text being searched, whereas in Chrome ctrl + F just doesn't work on the traces page.

Why I cannot use aggregate in zipkin-web

When I click aggregate in zipkin-web,the page will always in running and cannot open the result. And I haven`t seen any Exception in console.
I wonder How can I use the aggregate function.

Deploy script

Right now deploying the open source version of Zipkin is a very manual process. We need to reduce that burden a bit. Capistrano script would probably do as a start.

Errors thrown by InetAddress.getLocalHost

When running the web server locally, I encountered the following error:

[info] Running com.twitter.zipkin.web.Main -f zipkin-web/config/web-dev.scala
Nov 24, 2012 12:56:02 AM com.twitter.logging.Logger log
INFO: Loading configuration
Nov 24, 2012 12:56:02 AM java.util.logging.LogManager$RootLogger log
FATAL: Error in config file: %s
java.net.UnknownHostException: birderator: birderator
    at java.net.InetAddress.getLocalHost(InetAddress.java:1438)
    at com.twitter.zipkin.config.ZipkinConfig$class.$init$(ZipkinConfig.scala:30)
    at Evaluator__web$2ddev_16de0e15bca3885a651af7857b6a8434a7dbff86$$anon$1.<init>((inline):23)
    at Evaluator__web$2ddev_16de0e15bca3885a651af7857b6a8434a7dbff86.apply((inline):23)
    at Evaluator__web$2ddev_16de0e15bca3885a651af7857b6a8434a7dbff86.apply((inline):1)
    at com.twitter.util.Eval.applyProcessed(Eval.scala:197)
    at com.twitter.util.Eval.apply(Eval.scala:167)
    at com.twitter.ostrich.admin.RuntimeEnvironment.loadConfig(RuntimeEnvironment.scala:230)
    at com.twitter.ostrich.admin.RuntimeEnvironment.loadRuntimeConfig(RuntimeEnvironment.scala:256)
    at com.twitter.zipkin.web.Main$.main(Main.scala:28)
    at com.twitter.zipkin.web.Main.main(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at sbt.Run.invokeMain(Run.scala:66)
    at sbt.Run.run0(Run.scala:59)
    at sbt.Run.execute$1(Run.scala:48)
    at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:52)
    at sbt.TrapExit$.executeMain$1(TrapExit.scala:33)
    at sbt.TrapExit$$anon$1.run(TrapExit.scala:42)
Caused by: java.net.UnknownHostException: birderator
    at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:866)
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1258)
    at java.net.InetAddress.getLocalHost(InetAddress.java:1434)
    ... 20 more

This is because InetAddress.getLocalHost is not equivalent to InetAddress.getHostByName("localhost"). I have a small branch that resolves this issue by using InetAddress.getHostByName("localhost")

Query by multiple key/value pairs

Sometimes it's useful to be able to search for multiple key/value pairs. For example in order to find traces with 500 errors for a particular url.

key=http.uri, value=/i/discovery.json
key=http.responsecode, value=500 Internal Server Error

scala 2.9.2

Zipkin isn't on 2.9.2, do you mind if I update it to 2.9.2?

ui: Buffer underflow

FAT [20121023-18:33:11.993] ZipkinWeb: java.nio.BufferUnderflowException
FAT [20121023-18:33:11.993] ZipkinWeb:     at java.nio.Buffer.nextGetIndex(Buffer.java:498)
FAT [20121023-18:33:11.993] ZipkinWeb:     at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:355)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedBinaryAnnotation.toJson(json.scala:19)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceTimeline$$anonfun$toJson$10.apply(json.scala:79)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceTimeline$$anonfun$toJson$10.apply(json.scala:79)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:44)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.TraversableLike$class.map(TraversableLike.scala:194)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.collection.mutable.ArrayBuffer.map(ArrayBuffer.scala:44)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceTimeline.toJson(json.scala:79)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceCombo$$anonfun$toJson$12.apply(json.scala:95)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceCombo$$anonfun$toJson$12.apply(json.scala:95)
FAT [20121023-18:33:11.993] ZipkinWeb:     at scala.Option.map(Option.scala:133)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.conversions.json$WrappedTraceCombo.toJson(json.scala:95)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.web.App$$anonfun$12$$anonfun$apply$23.apply(App.scala:236)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.zipkin.web.App$$anonfun$12$$anonfun$apply$23.apply(App.scala:235)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$map$1$$anonfun$apply$10.apply(Future.scala:503)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Try$.apply(Try.scala:13)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$.apply(Future.scala:98)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$map$1.apply(Future.scala:503)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$map$1.apply(Future.scala:503)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:477)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:476)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Promise$$anonfun$transform$1.apply(Future.scala:883)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Promise$$anonfun$transform$1.apply(Future.scala:879)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Promise$$anonfun$respondWithoutChaining$1.apply(Future.scala:868)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.util.Promise$$anonfun$respondWithoutChaining$1.apply(Future.scala:863)
FAT [20121023-18:33:11.993] ZipkinWeb:     at com.twitter.concurrent.IVar$$anonfun$runqs$1.apply$mcV$sp(IVar.scala:174)
FAT [20121023-18:33:11.993] ZipkinWeb:     (...more...)

Why I start zipkin webUI, it throws a Unknown method exception

When I start web UI and query the trace log, it throws a Unknown method exception
Unsafe is a class in rt.jar. But I try all JDK such as JDk 7u17,openJDK7,JDK6u22,openJDK6 , I still cannot resolve this problem.
I wonder this sun.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V In which JDK version .

FAT [20130402-13:28:03.663] ZipkinWeb: org.apache.thrift.TApplicationException: Internal error processing getTraceSummariesByIds: 'java.lang.NoSuchMethodError: sun.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V'
FAT [20130402-13:28:03.663] ZipkinWeb: at org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.zipkin.gen.ZipkinQuery$FinagledClient.decodeResponse(ZipkinQuery.scala:5263)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.zipkin.gen.ZipkinQuery$FinagledClient$$anonfun$getTraceSummariesByIds$1.apply(ZipkinQuery.scala:5480)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.zipkin.gen.ZipkinQuery$FinagledClient$$anonfun$getTraceSummariesByIds$1.apply(ZipkinQuery.scala:5479)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:477)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:476)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Promise$$anonfun$transform$1.apply(Future.scala:883)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Promise$$anonfun$transform$1.apply(Future.scala:879)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Promise$$anonfun$respondWithoutChaining$1.apply(Future.scala:868)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.util.Promise$$anonfun$respondWithoutChaining$1.apply(Future.scala:863)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar$$anonfun$runqs$1.apply$mcV$sp(IVar.scala:174)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar$LocalScheduler.run(IVar.scala:125)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar$LocalScheduler.apply(IVar.scala:105)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar.runqs(IVar.scala:169)
FAT [20130402-13:28:03.663] ZipkinWeb: at com.twitter.concurrent.IVar.set(IVar.scala:311)

Prefix or regex lookups for annotations

We've received numerous requests for prefix lookups or even regex matching for annotations when looking up traces. This would allow users to match a prefix to a memcache key for example.

bin/sbt script throws an error

When running the bin/sbt script I get this error.
bin/sbt: line 16: [: too many arguments

It works as expected though, but we should fix the problem anyway.

ZipkinSpec doesn't tear down zookeeper instance

When running tests for zipkin server, the ZipkinSpec doesn't close the ZK instance that's started. This causes another run of the tests to fail since the ZK instance is still bound to the port.

To reproduce:

  $ bin/sbt
  > test // this may pass
  > ...
  > test // ZipkinSpec will fail

Merge script fails

I attempted to merge a pull request, but it seems our little merge scripts is broken somehow.

[marburg zipkin (master)]$ bin/git-pull-request.rb merge 245
Executing: 'git config --get github.token'
warning: peer certificate won't be verified in this SSL session
Executing: 'git config --get github.token'
warning: peer certificate won't be verified in this SSL session
bin/git-pull-request.rb:187:in `merge': Not merging into master (RuntimeError)
from bin/git-pull-request.rb:306

How can I quickly try out a zipkin demo?

I just wanna try out a zipkin demo to check whether it is suitable for my own project. I follow the Install page , and I have already built the zipkin project successfully by using the command bin/sbt update package-dist, it shows no errors and all tests are passed. So, what's the next, I can not find a dist directory to execute scp dist/zipkin*.zip [server].

I just wanna see a demo to demonstrate the functionality of zipkin. How can I try out the Zipkin UI(maybe the zipkin-web directory, I am not sure) to see the rails app?

Maybe the Install Page is out of date?

Counter logs successful send despite failure when SLF4J logger missing

When there's no SLF4J logger implementation on the classpath, the Zipkin log_span counter logs a successful send when it's actually failing due to a NoClassDefFoundException. Using a simple console StatsReceiver with no logger implementation, the output is as follows:

Counter HttpServer:connects = 1
Counter HttpServer:received_bytes = 149
Counter HttpServer:requests = 1
Stat HttpServer:handletime_us = 61161.0
Stat HttpServer:request_latency_ms = 4.0
Counter HttpServer:success = 1
Counter zipkin:create_log_entries:error:java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder = 1
Counter HttpServer:sent_bytes = 19
Stat HttpServer:connection_received_bytes = 149.0
Stat HttpServer:connection_sent_bytes = 19.0
Stat HttpServer:connection_duration = 170.0
Stat HttpServer:connection_requests = 1.0
Counter zipkin:log_span:ok = 1

... but the span is not actually sent to Scribe.

Document implementing a tracer

We should add a section to the readme describing how to implement a tracer for a protocol/library. Finagle and Thrift is probably the easiest example to describe.

Simplify configuration

We've had quite a bit of feedback via the mailing lists that the configurations of the collector, query service, and web UI are quite confusing. Users are unclear which parts they do and do not need, and are often put off by there being too many moving parts. In reality, the basic collector/query setup only requires a backing store (either Cassandra or Redis at the time of this writing).

The base configs are pretty confusing, so I'm proposing moving to a Finagle Builder style config.
Base config:

CollectorBuilder(Scribe.Interface)
  .writeTo(Cassandra.static("hostname"))
  .writeTo(Cassandra.zookeeperServerSet("/zookeeper/path/to/cassandra/cluster/serverset"))

Or maybe you want to use Redis instead

CollectorBuilder(...)
  .writeTo(Redis.static(...))

Or maybe you want to listen for JSON on an HTTP port (doesn't exist yet, but is conceivable)

CollectorBuilder(Http.Json)

Or, if you want something more intense with sampling

CollectorBuilder(...)
  .sampler(Sampler.static(0.5)) // sample 50% of spans

Or even adaptive sampling

CollectorBuilder()
  .sampler(AdaptiveSampler.zookeeper(...))

The builders would be similar for the query service and the web UI. This would also allow for users to spin up all three services in the same JVM instance by returning a Seq[Builder]

Thoughts?

Cleanup modules

Currently we have a confusing hierarchy of modules for the collector and query services including:

  • zipkin-common: common classes shared between the two
  • zipkin-thrift: Thrift structs and interfaces for both services
  • zipkin-scrooge: module to generate Scala from Thrift via Scrooge
  • zipkin-scribe: scribe specific classes for collector, including the service
  • zipkin-server: combination of code for collector and query, including the query service

I think we can clean this up by breaking up the latter two modules and make things more composable.

  • zipkin-cassie: implementation of Aggregates, Index, and Storage interfaces that use Cassandra as the backing store
  • zipkin-collector-core: core code for the collector service
  • zipkin-collector-scribe: scribe specific classes
  • zipkin-collector-service: module to compose everything together
  • zipkin-query-core: core code for query service
  • zipkin-query-service: module to compose everything together

is support finagle 6.3 & scala 2.10?

my demo project under finagle 6.3 & scala 2.10, i try connect zipkin but get IncompatibleClassChangeError

here is my code

class AskServiceConfig extends ServerConfig[AskService.ThriftServer] {

 var tracerFactory: Tracer.Factory = ZipkinTracer.apply(scribeHost = "127.0.0.1", scribePort = 9410, sampleRate = 1.0f )
FAT [20130415-09:27:00.076] Ask: A server service  threw an exception
FAT [20130415-09:27:00.076] Ask: java.lang.IncompatibleClassChangeError: Implementing class
FAT [20130415-09:27:00.076] Ask:     at java.lang.ClassLoader.defineClass1(Native Method)
FAT [20130415-09:27:00.076] Ask:     at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
FAT [20130415-09:27:00.076] Ask:     at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
FAT [20130415-09:27:00.076] Ask:     at java.security.AccessController.doPrivileged(Native Method)
FAT [20130415-09:27:00.076] Ask:     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
FAT [20130415-09:27:00.076] Ask:     at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
FAT [20130415-09:27:00.076] Ask:     at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.scribe$log_args.write(scribe.scala:203)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.scrooge.FinagleThriftClient$$anonfun$encodeRequest$1.apply(FinagleThriftClient.scala:27)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.scrooge.FinagleThriftClient$$anonfun$encodeRequest$1.apply(FinagleThriftClient.scala:22)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.util.Try$.apply(Try.scala:13)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.util.Future$.apply(Future.scala:56)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.scrooge.FinagleThriftClient$class.encodeRequest(FinagleThriftClient.scala:22)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.scribe$FinagledClient.encodeRequest(scribe.scala:290)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.scribe$FinagledClient.log(scribe.scala:305)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.logSpan(RawZipkinTracer.scala:123)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.mutate(RawZipkinTracer.scala:145)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.annotate(RawZipkinTracer.scala:217)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.record(RawZipkinTracer.scala:156)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.zipkin.thrift.ZipkinTracer.record(ZipkinTracer.scala:79)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.tracing.Trace$$anonfun$record$1.apply(Trace.scala:192)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.tracing.Trace$$anonfun$record$1.apply(Trace.scala:192)
FAT [20130415-09:27:00.076] Ask:     at scala.collection.immutable.Set$Set1.foreach(Set.scala:86)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.tracing.Trace$.record(Trace.scala:192)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.tracing.Trace$.record(Trace.scala:224)
FAT [20130415-09:27:00.076] Ask:     at com.twitter.finagle.thrift.ThriftServerTracingFilter$$anonfun$apply$2$$anonfun$apply$3.apply(ThriftServerFramedCodec.scala:173)
FAT [20130415-09:27:00.076] Ask:     (...more...)

Get links of traces

Is there a way to get the URL to a specific trace? So, having a trace id, how do I programmatically get the URL to the web page of that trace?

Thanks,
Derek

Try using Travis CI Again

I think it should be a goal of the project to be running on some type of CI system.

We should try to get things running on Travis CI again if possible.

sbt update failed

I tried to insall zipkin, when run

bin/sbt update package-dist

got the error info below,

[info] Resolving commons-httpclient#commons-httpclient;3.1 ...
[info] Resolving commons-logging#commons-logging;1.0.4 ...
[info] Resolving commons-codec#commons-codec;1.2 ...
[info] Resolving org.scala-tools.sbt#completion_2.9.1;0.11.2 ...
[info] Resolving jline#jline;0.9.94 ...
[info] Resolving org.scala-tools.sbt#run_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#task-system_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#tasks_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#tracking_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#cache_2.9.1;0.11.2 ...
[info] Loading project definition from /home/qinyuchun/source/zipkin-setup/zipkin/project
[info] Updating {file:/home/qinyuchun/source/zipkin-setup/zipkin/project/}default-64d38a...
[info] Resolving com.twitter#sbt-package-dist;1.0.5 ...
[info] Resolving ivysvn#ivysvn;2.1.0 ...
[info] Resolving org.markdownj#markdownj;0.3.0-1.0.2b4 ...
[info] Resolving org.freemarker#freemarker;2.3.16 ...
[info] Resolving com.twitter#sbt11-scrooge;3.0.0 ...
[info] Resolving com.twitter#sbt-thrift2;0.0.1 ...
[info] Resolving com.eed3si9n#sbt-assembly;0.8.2 ...
[info] Resolving org.scala-tools.sbt#sbt_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#main_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#actions_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#classfile_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#io_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#control_2.9.1;0.11.2 ...
[info] Resolving org.scala-lang#scala-library;2.9.1 ...
[info] Resolving org.scala-tools.sbt#interface;0.11.2 ...
[info] Resolving org.scala-tools.sbt#logging_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#process_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#classpath_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#launcher-interface_2.9.1;0.11.2 ...
[info] Resolving org.scala-lang#scala-compiler;2.9.1 ...
[info] Resolving org.scala-tools.sbt#incremental-compiler_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#collections_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#api_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#persist_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbinary#sbinary_2.9.0;0.4.0 ...
[info] Resolving org.scala-tools.sbt#compile_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#ivy_2.9.1;0.11.2 ...
[info] Resolving org.apache.ivy#ivy;2.2.0 ...
[info] Resolving com.jcraft#jsch;0.1.31 ...
[info] Resolving org.scala-tools.sbt#testing_2.9.1;0.11.2 ...
[info] Resolving org.scala-tools.testing#test-interface;0.5 ...
[info] Resolving org.scala-tools.sbt#compiler-interface;0.11.2 ...
[info] Resolving org.scala-tools.sbt#precompiled-2_8_1;0.11.2 ...
[info] Resolving org.scala-tools.sbt#precompiled-2_8_0;0.11.2 ...
[info] Resolving org.scala-tools.sbt#precompiled-2_9_0;0.11.2 ...
[info] downloading http://maven.twttr.com/com/twitter/sbt-thrift2_2.9.1_0.11.2/0.0.1/sbt-thrift2-0.0.1.jar ...
[warn] [FAILED ] com.twitter#sbt-thrift2;0.0.1!sbt-thrift2.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/com/twitter/sbt-thrift2_2.9.1_0.11.2/0.0.1/sbt-thrift2-0.0.1.jar. Please retry. (52886ms)
[warn] [FAILED ] com.twitter#sbt-thrift2;0.0.1!sbt-thrift2.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/com/twitter/sbt-thrift2_2.9.1_0.11.2/0.0.1/sbt-thrift2-0.0.1.jar. Please retry. (52886ms)
[warn] ==== twitter.com: tried
[warn] http://maven.twttr.com/com/twitter/sbt-thrift2_2.9.1_0.11.2/0.0.1/sbt-thrift2-0.0.1.jar
[info] downloading http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar ...
[warn] [FAILED ] ivysvn#ivysvn;2.1.0!ivysvn.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar. Please retry. (54377ms)
[warn] [FAILED ] ivysvn#ivysvn;2.1.0!ivysvn.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar. Please retry. (54377ms)
[warn] ==== twitter.com: tried
[warn] http://maven.twttr.com/ivysvn/ivysvn/2.1.0/ivysvn-2.1.0.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: FAILED DOWNLOADS ::
[warn] :: ^ see resolution messages for details ^ ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: ivysvn#ivysvn;2.1.0!ivysvn.jar
[warn] :: com.twitter#sbt-thrift2;0.0.1!sbt-thrift2.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[error] {file:/home/qinyuchun/source/zipkin-setup/zipkin/project/}default-64d38a/*:update: sbt.ResolveException: download failed: ivysvn#ivysvn;2.1.0!ivysvn.jar
[error] download failed: com.twitter#sbt-thrift2;0.0.1!sbt-thrift2.jar
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?

Display url in the lookup results

On the trace lookup page we can display the url of each trace in the list to make it easier to pick interesting ones. We would have to make the kv annotations that represent the url configurable as different instrumented libraries might call them differently.

Start scripts

We don't have any easy way of starting up the open source version. Need to add some scripts for that.

getTraceIdsByAnnotation

test in CassandraIndexSpec, why is

      // fetch by time based annotation, find trace
      var seq = cassandraIndex.getTraceIdsByAnnotation("service", "custom", None, 0, 3)()
      //seq mustEqual Seq(span1.traceId)

commented out? It seems like it isn't time based (the duration hasn't been indexed) but also like it should be found regardless.

also, what does the test:

      // should not find any traces since the core annotation doesn't exist in index
      seq = cassandraIndex.getTraceIdsByAnnotation("service", "cs", None, 0, 3)()
      seq.isEmpty mustBe true

mean? Why shouldn't the core annotation exist in index, and what does core annotation mean?

I'm implementing this for redis, and it seems like the reverse should be true, that the second should be commented out, and the first should be uncommented.

CassandraAggregates Futures

As Johan mentions, you shouldn't have to block on the remove in the storage methods, but instead chain the futures so you can return one that executes this first and then the batch.

bin/sbt update package-dist fails to resolve elephant-bird-cascading dependency

[info] Resolving com.twitter.elephantbird#elephant-bird-cascading2;3.0.0 ...
[warn] module not found: com.twitter.elephantbird#elephant-bird-cascading2;3.0.0
[warn] ==== ibiblio: tried
[warn] http://mirrors.ibiblio.org/pub/mirrors/maven2/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== twitter.com: tried
[warn] http://maven.twttr.com/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== powermock-api: tried
[warn] http://powermock.googlecode.com/svn/repo/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== scala-tools.org: tried
[warn] http://scala-tools.org/repo-releases/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== testing.scala-tools.org: tried
[warn] http://scala-tools.org/repo-releases/testing/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== oauth.net: tried
[warn] http://oauth.googlecode.com/svn/code/maven/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== download.java.net: tried
[warn] http://download.java.net/maven/2/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== atlassian: tried
[warn] https://m2proxy.atlassian.com/repository/public/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== jboss: tried
[warn] http://repository.jboss.org/nexus/content/groups/public/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== local-lookup: tried
[warn] file:/Users/arul/.m2/repository/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== local: tried
[warn] ==== elephant-bird repo: tried
[warn] http://oss.sonatype.org/content/repositories/comtwitter-105/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[warn] ==== Concurrent Maven Repo: tried
[warn] http://conjars.org/repo/com/twitter/elephantbird/elephant-bird-cascading2/3.0.0/elephant-bird-cascading2-3.0.0.pom
[info] Resolving org.scala-tools.testing#specs_2.9.1;1.6.9 ...
[info] Resolving org.scala-lang#scala-compiler;2.9.1 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.twitter.elephantbird#elephant-bird-cascading2;3.0.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[error] {file:/Users/arul/twitter/zipkin/}zipkin-hadoop/*:update: sbt.ResolveException: unresolved dependency: com.twitter.elephantbird#elephant-bird-cascading2;3.0.0: not found

Is this dependency publicly available?

The sample rate maybe wrong

The sample rate is 2, 4, 8, 16 ... 1024 in google Dapper, but in the finagle source, I find the sample value is a float value and can be set by any rate. I thought that may lose some feature in zipkin.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.