Code Monkey home page Code Monkey logo

cougar's Introduction

Cougar

Welcome to Cougar!

Cougar is a framework for making building network exposed service interfaces easy. It supports multiple paradigms of interface interaction over multiple network transports and protocols. It has an asynchronous core, supporting high throughput, highly concurrent implementations, and yet offers simpler synchronous interfaces when you want to trade speed for simpler code. It also comprises an interface definition language, client and server stub generator, plus a whole bunch of other goodies to make your life as simple as possible and to free you up to implement your business logic.

Read more on Cougar's site.

Project admin

Project tasks

Sonar report

Build Status

Licensing & Copyright

Main project: Copyright 2013-2014, The Sporting Exchange Limited. Specific files may differ.

Cougar is covered by "The Apache License, Version 2.0":

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Neither the copyright notices nor the license may removed or changed except with permission of the copyright holders or in compliance of the license terms.

cougar's People

Contributors

atropineal avatar eswdd avatar jorgemsrs avatar manjunathshivakumar avatar paulo-santos avatar richardqd avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cougar's Issues

Exclude commons logging dependencies in cougar dependencies

Cougar has a dependency on jcl-over-log4j, which causes problems when co-existing on classpath with commons-logging (since both expose same classes). We should exclude commons-logging when depending on libraries that have transitive dependencies to it.

Time limit to process requests

Add capability for a per-op time limit on the server side after which the EV processing thread should be interrupted.

Document administration interface

Doco to explain you shouldn't expose admin interface to public

Explain the level of XSS protection offered to plugins and to suggest you write plugins cognitive of this.

Switch to using classpath resources for PropertyConfigurer

We have an issue with cougar platform and could use your assistance.

When running any of our (non-Cougar) applications, we need to override cougar’s com.betfair.cougar.util.configuration.PropertyConfigurer or the application will not run.

The problem we are facing is related to the defaultConfig setter (PropertyConfigurer.setDefaultConfig). This setter is used by Spring to inject the resource in four different xml files, in which the resource is contained in a sub-folder named “conf”. As the path is given as a String starting at the root of the jar (e.g. ), the path is resolved as a ServletContextResource (due to the ‘/’), instead of a ClassPathResource. In light of this, the application fails during startup as a FileNotFoundException is thrown (i.e. such ServletContextResource does not exist).

In order to fix this problem, we are overriding the class (PropertyConfigurer) and altering the setDefaultConfig method to take a String (instead of a Resource), resolving the String ourselves (i.e. this.defaultConfig = new DefaultResourceLoader().getResource(defaultConfig)). This solution does not require any changes in the xml files.

A different (possible) solution to this problem would be to include a “classpath:” prefix on the four xml files (in the defaultConfig property location):
(cougar-core-impl-2.9.6.jar) conf.core-config-cougar-core-impl.xml
(cougar-client-2.9.6.jar) conf.cougar-application-spring.xml
(cougar-marshalling-impl-2.9.6.jar) module-config-cougar-marshalling.xml
(cougar-standalone-ev-2.9.6.jar) module-config-cougar-standalone-ev.xml.

Cougar client binary reconnect rate with wrong cert

While we were testing our app's binary connection to another , we configured it with the wrong cert for the service. This results in the following message being logged about 30 times a second or something crazy on our app:

2013-09-20 15:34:52.292: com.betfair.cougar.client.socket.IoSessionFactory INFO - Connected to ***:9003
2013-09-20 15:34:52.293: com.betfair.cougar.netutil.nio.RequestResponseManagerImpl INFO - Notified 0 outstanding requests for session 0023870
2013-09-20 15:34:52.335: com.betfair.cougar.client.socket.ExecutionVenueNioClient WARN - Unexpected exception from session 0023871
javax.net.ssl.SSLHandshakeException: SSL handshake failed.
        at org.apache.mina.filter.SSLFilter.messageReceived(SSLFilter.java:416)
        at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:299)
        at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilterChain.java:53)
        at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:648)

Because the SSL stuff is started after the connection is initiated because the original protocol didn’t support SSL, then it’s an exception handler in the client which triggers a reconnect. So there is no delay like the backoff and retry that occurs with the ReconnectTask in IoSessionFactory.

It would be hard to add one because there’s no context of the original connect request when the exception comes through. We could potentially keep a map of last attempted connect time by address (perhaps with a ttl on entries so we don’t end up with loads) and then implement the backoff logic based on that, but like I said, it’s not going to be an easy fix..

Aliases

We'd like to be able to "alias" a path so that a request for say "/webping" actually gets re-routed to "/customContext/health/v3/summary"

CO connection/heap coelescing

We want to add the capability such that a client can choose to migrate a subscription to a particular heap uri to a different server. This situation might occur if a request to subscribe comes in, the uri returned by the server matches one that another server has already returned, and so the client would prefer to receive and process only one stream of updates rather than many copies of the same thing.

We would probably want to do this as part of the subscription process, and should be toggleable.

It's also likely that the client would want to be able to direct the initial subscription request to the same server where the parameters (or a certain subset of parameters) for a particular CO method match previous requests - essentially some kind of subscription request to server affinity.

QoS - Client specified expiry

From #19:

Client timestamps in calls, EV answers with suitable code if max-age/expiry time is reached (effectively drops internal requests that have sat in the surge queue or the send buf (bin)) for too long
Allow clients to specify an inbound operation time that the client would specify – that way, we don’t bother to accept a call if it has already aged while it has been in the surge queue

Get integration tests running in travis

Also cleanup the upload of the test output when they fail to S3, and stop that running when testing pull requests (we could perhaps cat the output of the failed tests then?)

Cougar logger inserts comma's while rendering collections

Cougar uses "," (comma) as separator while rendering collections. This leads to request logs of variable field length which will break any downstream log parser tools, such as ones for importing into a database.

We should use the pipe character as the default.

Suppress code generation messages unless maven is in debug mode

Messages like these make up 95% (guessing) of the build output.

[INFO] Writing MapOfComplexOperation using template rescript/requestDataType.ftl to file /home/travis/build/betfair/cougar/baseline-client/client-idd/target/generated-sources/java/com/betfair/baseline/v2/rescript/MapOfComplexOperationRequest.java

I've never gained any value from these messages, and their volume makes troubleshooting builds un-necessarily harder. Proposal: suppress them unless mvn is invoked with -X. This will have a small but appreciated effects for folks that have to look at Cougar/Cougar-application build logs on a regular basis.

Implement the builder pattern for generated datatypes

Currently maven-idltods-plugin generates the code by following the Java beans pattern. A builder pattern would enable the created objects to be immutable and provide a more elegant solution for their construction.

CO connection failover

Currently, when a connection which has heap subscriptions is closed (for whatever reason), all heap subs are terminated and need to be manually reconstructed by the user of the client.

We want to add the capability to automatically failover to another server. This would necessitate:

  • Notifying the sub listeners that we're reconnecting
  • Resending the sub request (possibly the same used in #38) to a new server
  • Repopulating the heap state based on the new initial state
  • Notifying the sub listeners that we're reconnected

It would be desirable to be able to perform reconnection on a set of subs at one time, although we need to ensure that the service is given the opportunity to reject these reconnections. We need to ensure that we log the restart of this sub.

There is a danger with a batch reconnect that we overload the server in question since identity resolution will occur for each request and cached identities are perhaps unlikely to exist.

It's perhaps possible that we'd want to maintain a heap with >1 server at all times (in contrast to #38) so that we don't need to receive a new heap at a busy failure point, but only need to worry about the sub creation (on both client and server) - a kind of passive heap that's kept waiting for HA purposes if you will.

Complete ActiveMQ transport

Current activemq transport was cobbled together at the point of open sourcing to provide a concrete jms transport impl.

This should be finished off and integrated into the integration tests.

Drop unecessary IDL validations

an IDL Must have at least one dataType definition. Now that we have primitive or void return types this validation step is no longer required

Negatively cache missing deserializers in binary transport

It should be a non breaking change to add parameters to a BSIDL operation response.

Functionally, that is the case and everything works as it should, but there is a performance impact... on older clients who do not know about the type of the new field, the Hessian deserializer gets asked for its class, 'misses' and throws an Exception. This Exception is then caught and logged.

This happens every time a call is made that returns a type that the client doesn't know about. In the case of a high volume service call, that means every single request/response will throw (and catch, and log and swallow) an exception, which is far from desirable, and has been measured at around 25 times slower than the happy path.

An enhancement is needed to introduce a negative cache, which will avoid the lookup cost and associated exception throwing and logging behavior.

The enhancement has been implemented already for Cougar 2.7, 2.8 and 2.9. I will submit a pull request at some point with the 3.0 changes.

Allow GETs on operations with complex types

  1. As an internal developer
    I want APIs to be easy to code against
    So that my dev/maintenance tasks are easier, and produce cleaner code

  2. As a internal/external developer/tester
    I want to be able to exercise APIs with a browser where possible
    So that I have low barriers to integration and testing

At the moment we have (2) at the expense of (1), e.g. one call has 21 parameters, so a call can look like this:

service.doSomething(null, null, null, something, null, null, null, ... (and so on).

To facilitate (1) and (2) simultaneously, allow GETs for complex types.

Possible implementations already discussed somewhat:

a) All items in non-aggregator types must be stringable and not have any naming collisions (or be qualified appropriately)
b) Put JSON object on query string

Also:

Not very readable (subjective):

service.executeQuery(true, 43843, “1234”, null, null, OrderType.EXECUTABLE, null, null);

Much more readable:

            Query req = new Query();
            req.setAllTransactions(true);
            req.setAccountId(43843);
            req.setReferenceNumber(“1234”)
            req.setOrderType(OrderType.EXECUTABLE);
            service.executeQuery(req);

Can be wrapped with sugar:

            service.executeQuery(
                        new Query()
                                    .allTransactions()
                                    .forAccountId(43843)
                                    .withReferenceNumber(“1234”)
                                    .ofOrderType(OrderType.EXECUTABLE)
                                    .build()
                        )
            );

Request objects can also be constructed in Spring, cloned, passed around easily, and so on.

Request objects also don’t suffer from the major (IMO) problem of breaking generated clients at compile time if I add an optional parameter. The way we are doing it, all clients have to add a ‘null’ onto all their calls.

Why don’t we use request objects in our IDDs I wonder? Is there a good reason?

The flat signature in an IDD is often so it can still be called in a browser easily, we have no way to have complex request objects represented in a GET request at the moment.

It would be nice to remove that limitation. Instead of generating a synthetic request for a flat IDD signature, I wonder whether it would be better to make a more OO IDD and use dot notation for specifying object properties on the URI:

?request.thing=1&request.otherThing=1,2,3,4&request.nestedThing.property=foo

Perhaps even to make this backwards compatible it could assume that if you don’t specify a full path that matches anything it should scan the hierarchy and insert the value in any property that matches by name (so thing=1 would still get injected into request).

Cleanup dependencies

 <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <version>1.2</version>
                <executions>
                    <execution>
                        <id>enforce</id>
                        <configuration>
                            <rules>
                                <DependencyConvergence/>
                            </rules>
                        </configuration>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                    </execution>s>
            </plugin>
    </plugins>

Don't pass through all cougar-client errors to service transport

The issue was that Cougar was returning the error code from a server dependency as the error code from the server rather than an internal error.

The chain:
Client -> Server -> ServerDependency

Responses:
Client gets NotFound <- Server gets NotFound <- ServerDepenedency returns NotFound

What would be nice:
Client gets InternalError <- Server gets NotFound <- ServerDependency returns NotFound
And the Server logs the response it gets from its dependency

Document security recommendations

Including only ever setting passwords via overrides.properties

Cert strength:
Mention that Cougar supports whatever JSSE supports by default. Pointer to doco on how to change the supported algo's within JSSE

QoS capability

Various requests merge into one:

  • We are looking for ways to implement a mechanism for rejecting requests that are coming for non-essential flows when we detect that a service has an out of ordinary load/response time. Impl by having a fully automatic mechanism that based on some rules to take the decision of blocking calls or to limit the number of calls based on flow priorities and/or agreed SLA’s.

Thoughts on how to implement:

Have a list of monitors (or a status aggregator) with thresholds that cause a trigger (seperate to impact on monitoring status), and then based on that execute pieces of logic to process a request at the entry to the EV (and maybe to it's Q), to allow rejection of requests with a reason.

Protocol binding toggles

Add binding toggles by property so that all bindings for a particular protocol can be enabled/disabled at start time.

Increase CI build speed by combining build and test steps

The initial build does an install and runs no tests, and then the second build runs all tests (including unit and integration).

We could combine these two builds, which might result in a faster build.

The downside being that it might take longer for the committer to become aware of a compilation failure.

Sensible client error when receive HTTP 413

we would like cougar client to be modified to return a sensible error when it receives an HTTP 413 (HEAD full) error from Jetty.

At the moment it returns:
DSC-0026, Server is NOT identified as a Cougar 2 Server

This error can be simulated by passing a very long URI Rescript request

Memory leak in JMSEventTransportImpl client

Looking into Cougar source code (EventPublisherRunnable class from JMSEventTransportImpl.java, sonic-transport module) I saw that the MessageProducer objects are never closed:
public void run() {
try {
Session session = sessionManager.get();
Destination destination = createDestination(session, destinationName);
MessageProducer messageProducer = session.createProducer(destination);
TextMessage textMessage = getEventMarshaller().marshallEvent(descriptor, event, session);
messageProducer.send(textMessage, textMessage.getJMSDeliveryMode(), textMessage.getJMSPriority(), textMessage.getJMSExpiration());
success = true;
} catch (CougarFrameworkException cfe) { // Catch possible exception thrown from session creation
success = false;
error = cfe;
} catch (JMSException ex) { // Catch any other exception thrown during message publication
success = false;
error = ex;
} finally {
unlock();
}
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.