Loading Appnexus Reports using Spring batch

Introduction to Appnexus reporting API

Unlike other Apis which works on request-response model, Appnexus reporting API is quite different as the report could not be directly downloaded, but has to be scheduled instead.  It has the following sequence of steps:

• Step 1. Create a JSON-formatted report request
• Step 2. POST the request to the Report Service
• Step 3. GET the report status from the Report Service
• Step 4. GET the report data from the Report Download Service

The following sequence diagram shows the steps involved in requesting and downloading a report from Appnexus.



Modelling the above sequence using Spring batch

Configuration XML

The following listing shows the sample configuration file that is required for creating the batch job.

<batch:job id="reportLoadJob">
 <batch:step id="reportStep1" next="reportStep2">
  <batch:tasklet ref="reportRequestTasklet" />
 </batch:step>
 <batch:step id="reportStep2" next="reportStep3">
  <batch:tasklet ref="reportStatusCheckTasklet" />
 </batch:step>
 <batch:step id="reportStep3" next="reportStep4">
  <batch:tasklet ref="reportDownloadTasklet" />
 </batch:step>
 <batch:step id="reportStep4">
  <batch:tasklet>
   <batch:chunk reader="cvsFileItemReader" 
    writer="mysqlItemWriter" commit-interval="2">
  </batch:chunk>
   </batch:tasklet>
 </batch:step>
</batch:job>

Step 1: Report Request Tasklet

This is the Tasklet that is responsible for hitting Appnexus reporting API with the required parameters, and receives the report ID that would be eventually generated by Appnexus.


Step 2: Report Status Check Tasklet

This is the Tasklet that is responsible for hitting Appnexus reporting API with the report ID that was fetched in the previous step to check whether the requested report has been generated. If not, the step is repeated again with some time interval such that it is then until the report is ready. Once the report is ready, you will receive a response with status okay, along with the download URL, which will be used by the subsequent step to download the report.


Step 3: Report Download Tasklet

In this Tasklet, the download URL which was received in the previous step will be used to download the created CSV file. The received a file is stored in the local file system and the name of the file is updated in the job execution context, such that the next step could read the file name from it.


Step 4: Report Persist Tasklet

This is the step that is responsible for reading the data from the CSV file and persisting it into the database. For this to be done, first of all an entity that represents each line of the file has to be created. Then they could use chunk oriented processing to save the data in the database. This Tasklet is different from all other Tasklet we have seen so far, because they here the Tasklet consist of a reader, a writer and commit interval.

The reader is responsible for reading the CSV file online at a time, and it is passed on to the writer in chunks specified by the commit interval, following which the writer will save the items to the database.

Thus is quite easy to model a multistep job using Spring batch. Kindly post a comment if you either like it or if you feel the content is inappropriate in some way. If you need help modelling any of your jobs using Spring batch, kindly post a comment, that will be taken up as a topic for the subsequent posts. 

Load Appnexus Country List using Spring Batch: Part 2

In the previous post we saw the configuration required for loading countries from Appnexus using Spring batch. In this post we will look at the implementation of the countryLoadTasklet and pagingAwareStepExecutionListener.

Country Load Tasklet

This class extends from a superclass AbstractPageLoadTasklet which is designed to accommodate any Tasklet which deals with pagable response.

@Scope("step")
public abstract class AbstractPageLoadTasklet
 <TYPE, REPO extends JpaRepository<TYPE, ?>, 
  RESPONSE extends AbstractPagableResponse>
 implements Tasklet {

 @Autowired
 private AppnexusRestClient appnexusClient;

 @Value("#{stepExecutionContext['" + StepAttribute.START_ELEMENT + "']}")
 private int startElement;

 @Override
 public RepeatStatus execute(StepContribution stepContribution,
 ChunkContext chunkContext) throws Exception {
 // ...
 }

 public abstract List<TYPE> 
 getItemsFromResponse(RESPONSE response);

 public abstract ResponseContainer<RESPONSE> 
 getPageableResponse(
  AppnexusRestClient appnexusClient, int startElement);

 public abstract REPO getRepository();

 protected void save(List<TYPE> items) {
 getRepository().save(items);
 }
}

Kindly pay attention to the fact that this is abstract class and there are a few methods which are specific to the Tasklet has been made abstract. The implementation of the class itself is very simple:

Detail 1: Implementing Tasklet interface

public abstract class AbstractPageLoadTasklet implements Tasklet {

It extends from Tasklet interface, which is provided by Spring batch.

Detail 2: Injection of AppnexusRestClient

@Autowired
private AppnexusRestClient appnexusClient;

Auto wiring of AppnexusRestClient, the creation of which was discussed in one of the previous post.

Detail 3: Injection of startElement

@Value("#{stepExecutionContext['" + StepAttribute.START_ELEMENT + "']}")
private int startElement;

An interesting observation to make here is the injection of startElement, which is actually being read from the step execution context. As we saw in the previous post, each time the step is being executed, the start element will be updated depending upon the page that has to be fetched. This ensures that the subsequent pages that are being fetched our progressive till we reached the end. The job of updating the start element at the end of each step is performed by pagingAwareStepExecutionListener the details of which are given below.

Detail 4: Implementation of the execute method

This method has three parts:

1. Reading the list of countries by using the appnexusClient. 

ResponseContainer<RESPONSE> responseContainer = 
 getPageableResponse(appnexusClient, startElement);
RESPONSE response = responseContainer.getResponse();
// fetch items from response
List<TYPE> items = getItemsFromResponse(response);

2. Updating the job execution context with the variables: 1.start element 2. Number of elements 3.Count (total number of elements) 

// Part 2: update execution context
ExecutionContext executionContext = chunkContext.getStepContext()
 .getStepExecution().getExecutionContext();
executionContext.put(StepAttribute.START_ELEMENT, response.getStartElement());
executionContext.put(StepAttribute.NUM_ELEMENTS, response.getNumElements());
executionContext.put(StepAttribute.COUNT, response.getCount());

3. Saving the fetch items into the database using the repository. 

// Part 3: save items
save(items);
return RepeatStatus.FINISHED;

CountryLoadTasklet 

The following listing show the implementation of the country Load Tasklet, which extends from AbstractPageLoadTasklet.

public class CountryLoadTasklet extends
  AbstractPageLoadTasklet<Country, CountryRepo, CountryListResponse> {

 @Autowired
 private CountryRepo countryRepo;

 @Override
 public List<Country> getItemsFromResponse(CountryListResponse response) {
  return response.getCountries();
 }

 @Override
 public ResponseContainer<CountryListResponse> getPageableResponse(
   AppnexusRestClient appnexusClient, int startElement) {
  return appnexusClient.getCountryList(startElement);
 }

 @Override
 public CountryRepo getRepository() {
  return countryRepo;
 }
}

Kindly pay attention to the implementation of all the abstract methods which are defined in the superclass. The implementation of these methods itself is simple, because there is no logic in the implementation of any of these methods, as the superclass has all the intelligence required to fetch the pages from Appnexus. 

PagingAwareStepExecutionListener

The following listing shows the implementation of this class.
Detail 1: Implementing StepExecutionListener interface 

@Component
public class PagingAwareStepExecutionListener implements StepExecutionListener {

 @Override
 public void beforeStep(StepExecution stepExecution) {
    // ...
 }

 @Override
 public ExitStatus afterStep(StepExecution stepExecution) {
    // ...
 }
}

StepExecutionListener interface is a component provided by Spring batch and listeners must implement this interface. This interface consists of two methods, beforeStep and afterStep. 

Detail 2: Implementation of beforeStep method:

This method is invoked before the execution of each step. The implementation is quite simple, which involves reading the start element from the job execution context, which were supplied by the use of before invoking the job and updating the value in the step execution context such that the step can read it to fetch the right page.

Details 3: Implementation of afterStep method:

This method performs the following:

1. Reads the paging information from the step execution context, which was populated by the step itself.

ExecutionContext context = stepExecution.getExecutionContext();
int startElement = context.getInt(StepAttribute.START_ELEMENT);
int count = context.getInt(StepAttribute.COUNT);
int numElements = context.getInt(StepAttribute.NUM_ELEMENTS);
int remainingElements = count - (startElement + numElements);

2. Checks if more items are to be read from Appnexus. If there are more items, update the start element in the step execution context accordingly and return CONTINUE as the exit status, otherwise returned END as exit status.

if (remainingElements <= 0) {
 System.out.println("Task finished.");
 return new ExitStatus("END");
}

stepExecution.getJobExecution()
 .getExecutionContext()
 .putInt(StepAttribute.START_ELEMENT, startElement + numElements);

return new ExitStatus("CONTINUE (Items Remaining: " 
 + remainingElements + ")");

Conclusion

We have seen the implementation details of the two most important components the Tasklet and the Execution Listener. We are also seeing the abstract class AbstractPageLoadTasklet, which contains the code logic hence reducing the implementation of CountryLoadTasklet to the minimum. In the next post we will design a service layer for the batch jobs.

Load Appnexus Country List using Spring Batch: Part 1

In this post we will discuss the details of loading the list of countries from Appnexus using Spring Batch.

Sequence of steps

The following diagram shows the sequence of steps involved in fetching the list of countries from Appnexus.

Summary of steps

1. Fetch the authentication token from Appnexus using the user name and password. 
2. Requesting country endpoint with start element zero. 
3. Identifying the remaining elements from the three values start element, num_elements and count. 
4. Requesting for subsequent elements using the same endpoint. 
5. Continue the process until all the pages are read. 

Using for loop 

The above interaction could be simply achieved by using a for loop or a while loop. But the object to this post is to present you with an alternate mechanism of achieving the same and the advantages of it. I have chosen Spring Batch for this purpose, in the rest of this post we will look at a brief introduction to Spring batch followed by the advantages of choosing it.

Introduction to Spring Batch 

Spring batch is a framework designed to provide the necessary infrastructure for the execution of batch jobs. For more information about Spring batch refer the Wikipedia entry or the official homepage.

Introduction to batch jobs 

For a job to be classified as a batch job it should possess a few characteristics. For instance,

• it should be long-running
• it should not need human interaction
• most often it will be run on a periodic basis. 

These are just a few characteristics that are vital for a batch job, for more information on batch jobs and their characteristics kindly refer to the following Wikipedia entry on batch jobs.

Why batch infrastructure? 

As I suggested in one of the above paragraphs, the specific problem of loading the list of countries from Appnexus could be solved just using a for loop, but then why should use Spring batch to do it. There are a couple of disadvantages in doing it with plain Java.

• No history of the jobs run. 
• No restart capability. 
• No protection against the job being running again. 
• Retry and failure handling has to be implemented ourselves. 
• In case of complex jobs with multiple steps, the flow has to be managad manually. 

Spring batch provides a lot of functionality like history of jobs, restarting capability, protection against unwanted reputation of jobs, error handling and custom step flow. Getting into more details about Spring batch is not the purpose of this post, kindly refer to the official documentation for more information.

Spring Batch Terminology

Job

A Job is an entity that encapsulates an entire batch process.

Step

A Step is a domain object that encapsulates an independent, sequential phase of a batch job. Therefore, every Job is composed entirely of one or more steps. A Step contains all of the information necessary to define and control the actual batch processing.

Tasklet

The Tasklet is a simple interface that has one method, execute, which will be a called repeatedly by the TaskletStep until it either returns RepeatStatus.FINISHED or throws an exception to signal a failure.

All the above definitions are taken from the official documentation, kindly refer to it for more information on Spring batch architecture and terminology.

Required Components

Going by the about definition, we could adopt the sequence listed in the above diagram as follows:

Appnexus country load Job - Overall flow of loading all the countries from Appnexus.

Appnexus country load Step - As you can see the job has 3 calls to Appnexus country endpoint, which is naturally the steps involved.

To summarise, Appnexus country load has one job, which has three steps, but then all the steps are similar (except for the start element which is different). So this leaves us with a single job with a single step which is repeatedly called until all the pages are read.

Spring Batch Configuration

Please find below the configuration file for the job we just discussed: country-load-job.xml

<batch:job id="countryLoadJob">
 <batch:step id="step1">
  <batch:tasklet ref="countryLoadTasklet" />
  <batch:next on="CONTINUE*" to="step1" />
  <batch:end on="END" />
  <batch:listeners>
   <batch:listener ref="pagingAwareStepExecutionListener" />
  </batch:listeners>
 </batch:step>
</batch:job>

As you can see here, the configuration is actually simple. As you can see from the XML, there is only one step, "step1", the step is being called repeatedly as long as the status is "CONTINUE" and the job is terminated once the status is "END". Kindly pay attention to the listener "pagingAwareStepExecutionListener" which determines the status of the step depending upon the values start element, number of elements and the total number of elements.

This brings us to the end of this post on loading Appnexus countrie using Spring batch. In the following post we shall get into the details of the implementation of countryLoadTasklet and pagingAwareStepExecutionListener.

Kindly post a comment if you either like it or if you feel the content is inappropriate in some way.

Consuming Appnexus Country Service (and similar Read Only Services)

In the previous post, we have seen introduction to Appnexus and the technologies required to consume Appnexus API through Java. We have also seen the authentication mechanism using user name and password, which is a prerequisite for consuming any service to API. In the last post, we have also created a central Authentication Service which will take care of managing the life cycle of the authentication token and automatically renewing it when it expires, such that all the consumers get a valid authentication token.

In this post, we are going to see how we could leverage all that has been learned so far to read the list of countries from Appnexus.

Introduction to Country Service 

Appnexus provides a list of read-only services using which we could consume all the Meta data that are required for certain other services. The read-only Country Service allows you to see information about the countries registered in the AppNexus system. You can use this service to retrieve the country codes for targeting in campaigns.

Objectives 

• Set up the necessary infrastructure that is required to consume the list of countries. 
• Automatically add the authentication token using the authentication service. 

Quoted from Appnexus:

The following is a sample request given from commandline to fetch the list of countries. For more information on country service refer to the following link. https://wiki.appnexus.com/display/api/Country+Service

$ curl -b cookies -c cookies 'http://api.appnexus.com/country'
 
{
    "response": {
        "status": "OK",
        "count": 250,
        "start_element": 0,
        "num_elements": 100,
        "countries": [
            {
                "id": 1,
                "name": "Anonymous Proxy",
                "code": "A1"
            },
            {
                "id": 2,
                "name": "Satellite Provider",
                "code": "A2"
            },
            {
                "id": 3,
                "name": "Other Country",
                "code": "O1"
            },
            {
                "id": 4,
                "name": "Andorra",
                "code": "AD"
            },
            {
                "id": 5,
                "name": "United Arab Emirates",
                "code": "AE"
            },
            {
                "id": 6,
                "name": "Afghanistan",
                "code": "AF"
            },
            ...
        ],
        "dbg_info": {            
        }
    }
}

Required Components 

First of all, we have to create the interface that specifies the contract of Appnexus Country Service. Pay attention to the @GET annotation which specifies the endpoint of Country Service. Also note that we are passing start_element as a request parameter for each request, the reason for passing it is that, countries are returned to us in pages, and when we are fetching the subsequent page the start_element has to be appropriately specified.

AppnexusRestClient.java

public interface AppnexusRestClient {

 @GET("/country")
 ResponseContainer getCountryList(
   @Query("start_element") int startElement);
   
 // ... getters, setters
}

Modelling Response Object 

The following listings shows all the classes that are required for modelling the response returned by Appnexus. If you could compare the structure of the below classes with the JSON returned by Appnexus in the above example you could see that we have just mirrored the same structure using Java.

The below classes the outermost container, that just encapsulates the response object. It doesn't serve any other purpose. Kindly pay attention to the fact that this response container could contain any type of response, that's why it is parameterized by using generics.

public class ResponseContainer<T> {

 private T response;
 
 // ... getters, setters
}

The reason for creating abstract response is that, the format of response object is constant for almost every response, instead of repeating the same field again and again in all the response objects, this class would help avoid repetition.

public abstract class AbstractResponse {

 private String status;
 private DbgInfo dbgInfo;

 private String errorId;
 private String error;
 private String errorDescription;
 
 // ... getters, setters

}

The reason for creating the below class is that a few services like the Country Service for example returned the response in pages. Pay attention to the start_element, num_element and count in the response from Appnexus. Whenever the response is pagable, you'll have these fields as part of the response. So based on the value of these elements, we should determine whether there are more elements to be fetched and give subsequent requests accordingly. By having the below class, whenever we are requesting for an element which should be fetched in pages, we could use it.

public abstract class AbstractPagableResponse extends AbstractResponse {

 private Integer count;
 private Integer startElement;
 private Integer numElements;

 // ... getters, setters
}

Finally the below classes are the ones that are specific to the Country Service. These classes are self-explanatory.

public class CountryListResponse extends AbstractPagableResponse {

 private List<Country> countries;
 
 // ... getters, setters

}

public class Country {

 private Long id;
 private String name;
 private String code;
 
 // ... getters, setters
 
}

The following listing shows the procedure to create a new rest adapter and to send a request to Appnexus, followed by reading the response. Read the comments in the listing to understand what is happening in each of the steps.

// 1. create new gson
Gson gson =  new GsonBuilder()
 .setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES)
    .create();
  
// 2. create a rest adapter  
RestAdapter restAdapter = new RestAdapter.Builder()
    .setEndpoint(appnexusApiEndpoint)
    .setRequestInterceptor(new RequestInterceptor() {

     @Override
     public void intercept(RequestFacade request) {
     
      // add authentication header here
      request.addHeader("Authorization",     authenticationService.getAuthenticationToken());
     }
    }).setConverter(new GsonConverter(gson)).build();

// 3. create a rest client
AppnexusRestClient client = restAdapter.create(AppnexusRestClient.class);

// 4. request for country list and get the response

int startElement = 0;
ResponseContainer responseContainer = client.getCountryList(startElement);

// 5. read the country list from response
List list = responseContainer.getResponse().getCountries();

Request Interceptor

In the above listing, in the second step you could see that a request interceptor is being added. Its purpose is to add the authorization header to the outgoing request.  This is exactly where the authentication token is fetched from the authentication service and it is being added to the request.

Retrieving Subsequent Pages

In the above samples, we have only seen how to retrieve the first page using a rest client. In the next post we will see how to consume all the available pages using Spring Batch.

Kindly post a comment if you either like it or if you feel the content is inappropriate in some way.

Creating a central Authentication Service for Appnexus

Introduction 

Before you can make calls to any AppNexus API service or report, you must use your username and password to get an authorization token. The token remains active for 2 hours, during which you do not need to re-authenticate.  Click here for more information about Appnexus Authentication Service.

Limitations 

The AppNexus API permits you to authenticate successfully 10 times per 5-minute period. Any subsequent authentication attempts within those 5 minutes will result in an error.

Objective 

• Create an Authentication Service which could be used by all the services. 
• Automatically renew the authentication token if it expires. 

Assumptions 

For the sake of this post, I'm assuming that you are using spring for dependency injection and the Authentication Service is being configured as Singleton.

AuthenticationService.java

public interface AuthenticationService {

 String getAuthenticationToken();

}

AuthenticationServiceImpl.java

@Component
public class AuthenticationServiceImpl implements AuthenticationService {

 private String authToken;
 private Date lastUpdated;

 private int validityInMinutes = 120;

 @Autowired
 private AuthenticationRequest authenticationRequest;

 @Autowired
 private AuthenticationClient authenticationClient;

 @Override
 public String getAuthenticationToken() {

  // if authToken is null, get new and update vars
  if (authToken == null || hasAuthTokenExpired()) {
   updateAuthToken();
  }

  return authToken;
 }

 private boolean hasAuthTokenExpired() {

  // check time, if more than validityInMinutes, return true

  DateTime last = new DateTime(lastUpdated);
  DateTime current = new DateTime();

  Period period = new Period(last, current);
  Minutes elapsed = Minutes.standardMinutesIn(period);

  if (elapsed.getMinutes() > validityInMinutes)
   return true;

  return false;
 }

 private void updateAuthToken() {

  ResponseContainer  authResponseContainer =     authenticationClient.authenticate(authenticationRequest);

  authToken = authResponseContainer.getResponse().getToken();
  lastUpdated = new Date();
 }
}

The above code is actually simple, it just follows the steps listed below:

1. Each time an authentication token is requested, a check is performed to see if the authentication token is null, if yes, then "updateAuthToken" is called, which will retrieve a fresh authentication token from Appnexus and update the variable "authToken", which will be returned to the caller. 
2. If an existing authentication token is found, then the last updated time is checked to make sure that the token is still valid. If yes, then the token is returned, otherwise "updateAuthToken" is called, which will take care of updating the token. 

Advantages

1. Using this method, the whole logic of maintaining and supplying a valid authentication token to all the consumers is encapsulated in a single service. 
2. This will also avoid the overhead of checking for authentication related errors in all the rest calls to Appnexus. 

Pay attention to the fact that "DateTime" has been used to calculate the elapsed duration. This belongs to joda date-time API. Kindly refer to the official documentation for more details.

Kindly post a comment if you either like it or if you feel the content is inappropriate in some way.

Creating an Authentication Rest Client using Retrofit for Appnexus

Before you can make calls to any AppNexus API service or report, you must use your username and password to get an authorization token. The token remains active for 2 hours, during which you do not need to re-authenticate. This page walks you through the authentication process.

For more information on Appnexus Authentication Service refer to the following wiki entry. Since you need a username and password to access the wiki, some part of it has been reproduced here just for the sake of introducing the concept.

This post has been structured into two parts:

Part 1 explains the authentication procedure for Appnexus.
Part 2 explains the Java constructs that are required to consume this API.

Part one

The following points outlines the steps involved in authenticating before consuming any service.

Step 1. Create a JSON-formatted file including your username and password
Step 2. POST the file to the authentication service, to retrieve the authentication token.
Step 3. Use the token when making calls to API services and reports

The first two steps are specific to authentication, and the third step just mentions that we have to supply the authentication token for subsequent communication.

Authentication Request Format

 
$ cat auth
{
    "auth": {
        "username" : "USERNAME",
        "password" : "PASSWORD"
    }
}

Authentication Response Format

 
$ curl -b cookies -c cookies -X POST -d @auth 'https://api.appnexus.com/auth'
{
    "response": {
        "status": "OK",
        "token": "h20hbtptiv3vlp1rkm3ve1qig0",
        "dbg_info": {
            "instance": "13.hbapi.prod.nym1",
            "slave_hit": false,
            "db": "master",
            "time": 165.15779495239,
            "start_microtime": 1332339041.1044,
            "version": "1.11.20"
        }
    }
}

Kindly pay attention to the structure of the JSON. There is an outermost object which encapsulates the actual "response" variable, which also contains the status, required authentication token and dbg_info.

Using the Authentication Token

 
$ curl -H "Authorization: 622cee5f8c99c81e87614e9efc63eddb" 'https://api.appnexus.com/member'
 
{
    "response": {
        "members": [
            {
                "id": "1",
                "name": "Car company c\/o Ad Agency",
                "active": "1",
                "bidder_id": "2"
            }
        ]
    }
}

Part two

Since we have already explained in the previous post that we will be using Retrofit to consume the API, in the following section I will present code samples related to it.

Modelling JSON response in Java

In order to read the response JSON received from Appnexus, we need a class in Java which mirrors the same structure as the incoming JSON. This has been done using two classes, namely:

ResponseContainer<T>.java

This class acts as an outermost container, which encapsulates the actual response. Pay attention to the use of generics in this class.

 
public class ResponseContainer<T> {

 private T response;

 public ResponseContainer() {
  super();
 }

 public T getResponse() {
  return response;
 }

 public void setResponse(T response) {
  this.response = response;
 }

 @Override
 public String toString() {
  return "ResponseContainer [response=" + response + "]";
 }

}

AuthenticationResponse.java

This class has the structure of the actual response which contains the authentication token. Also note that, this class extends from AbstractResponse.java which contains the dbg_info, shown above. Since certain fields like status and dbg_info are common to all the responses, we have created this class, from which every other response could extend.

 
public class AuthenticationResponse extends AbstractResponse {

 private String token;

 public String getToken() {
  return token;
 }

 public void setToken(String token) {
  this.token = token;
 }

 @Override
 public String toString() {
  return "AuthenticationResponse [token=" + token + "]";
 }

}

Modelling JSON request in Java

Before looking into the details of the rest client itself, we should also model the request object as well. This is again similar, as the object has to mirror the request JSON. I have not enclosed the class here, but it should be quite straightforward to create one.

Retrofit Authentication Rest Client Interface

The following code sample outlines the interface that is required to talk to Appnexus using Retrofit. Pay attention to the use of @POST annotation using which the endpoint has been specified. Also observe how the request objects and the response objects we just modelled has been used.

The biggest advantage of using Retrofit is that, the contract is extremely clear and it only deals with the business in question. It totally hides away the nuances of creating HTTP related infrastructure.

 
public interface AuthenticationRestClient {

 @POST("/auth/")
 ResponseContainer<authenticationresponse> authenticate(
   @Body AuthenticationRequest authenticationRequest);
}

Creating the rest client and using it

Finally, once everything has been done, we could create an instance of AuthenticationRestClient using the following code block, and start interacting with Appnexus to fetch the authentication token.

 
RestAdapter restAdapter = new RestAdapter.Builder()
 .setEndpoint(appnexusApiEndpoint).setLogLevel(LogLevel.HEADERS).build();

// create auth client
AuthenticationRestClient authClient = restAdapter.create(AuthenticationRestClient.class);
ResponseContainer<authenticationresponse> responseContainer = authClient.authenticate(authenticationRequest);

// get the token here
String token = responseContainer.getResponse().getToken();

Conclusion

I could see that a lot of code samples has been given, they should provide enough guidance to understand the concepts outlined, if you have any difficulty in understanding how the pieces fit together, do drop a comment with the details, I'll try to clarify them. In the following post, I'll outlined the following details:

• Details of automatically adding the authentication token in the subsequent requests.
• Mechanisms for the using the same authentication token in all the subsequent requests until it expires and also ways to retrieve it again after it expires.

Consuming Appnexus API using Java

Introduction to Appnexus 

Appnexus is the leading demand-side and supply-side platform which specializes in RTB. For more information on Appnexus kindly refer to this Wikipedia entry. 

About Me 

I am working for a media company which specialises in performance trading and most of the campaigns are run through Appnexus apart from other leading AdExchanges. As part of the software platform, we consume Appnexus services using the API provided by Appnexus. In this series of posts I'll show you the technology used and the architecture required to consume Appnexus API. 

Audience 

These posts are targeted at software engineers developing applications which consume Appnexus services using their API or similar third party services. I also assume a fair understanding of Java and related web technologies. 

Restful Web services 

Appnexus exposes all of its services as restful Web services. REST stands for Representational State Transfer, which in short could be put forward as the method of consuming third-party services using HTTP protocol. For more information on REST, please refer to the following Wikipedia entry. 

Consuming Restful Services 

There are a couple of options available for consuming restful services. Namely,

• Apache HTTP client
• Spring Rest template
• Square Retrofit

Since using Apache HTTP client is of the most straightforward methods of consuming restful Web services, most of the developers would tend to choose it. On the other hand, a couple of developers might also choose Spring rest template-based implementation in order to consume the services. I would suggest using Square's Retrofit for that purpose. The following paragraphs contain code examples for consuming an example service using each of the above methods, which will help me highlight the differences between each of these methods and also to put forward the fact that Retrofit is better in this specific case. 

Using Apache HTTP Client:

HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("http://www.github.com");
HttpResponse response = client.execute(request);

// Get the response
BufferedReader rd = new BufferedReader
  (new InputStreamReader(response.getEntity().getContent()));
    
String line = "";
while ((line = rd.readLine()) != null) {
  textView.append(line); 
} 

Using Spring Rest-template

  HttpHeaders headers = new HttpHeaders();
    headers.setContentType(MediaType.APPLICATION_JSON);
    headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));

    RestTemplate template = new RestTemplate();

    HttpEntity<String> requestEntity = new HttpEntity<String>(
        RestDataFixture.standardOrderJSON(),headers);

    ResponseEntity<Order> entity = template.postForEntity(
        "http://localhost:8080/aggregators/orders",
        requestEntity, Order.class);

    String path = entity.getHeaders().getLocation().getPath();

    assertEquals(HttpStatus.CREATED, entity.getStatusCode());
    assertTrue(path.startsWith("/aggregators/orders/"));
    Order order = entity.getBody();

Using Retrofit:

public interface GitHubService {
  @GET("/users/{user}/repos")
  List<Repo> listRepos(@Path("user") String user);
}

RestAdapter restAdapter = new RestAdapter.Builder()
    .setEndpoint("https://api.github.com")
    .build();

GitHubService service = restAdapter.create(GitHubService.class);
List<Repo> repos = service.listRepos("octocat");

There is a fundamental difference between each of these three methods. 

1. With Apache HTTP client, the response is received as a string, following which we have two manually deserialize it. Explaining serialization and deserialization is out of the scope of this post, kindly refer to the following Wikipedia entry for more information. 

2. When Spring Rest-client is used, we get a response object, and which encapsulates the required entity, which we could fetch by invoking a method on the response. Though this method is simple, it is not elegant. If you pay attention to the construction of the client, you could notice that it doesn't really represent what is being consumed.

template.postForEntity("http://localhost:8080/aggregators/orders", requestEntity, Order.class);

The specific line that has been enclosed above does not represent the business in hand, while it is more technical. 

3. Finally, if you take a look at the way the retrofit client has been constructed, the purpose of it is clear. Using the following line

restAdapter.create(GitHubService.class);

an implementation of GitHubService.class is created at run-time, following which we could use GitHubService just like any other services that are defined locally. Once such a client has been declared and instantiated the rest of the application we don't worry about the fact that the services are remote. 

Conclusion 

This is not a post to compare different technologies, because each of these technologies mentioned above have their own place in the technical world. I am suggesting that considering the use case in hand, Retrofit would be a better fit. Since we have seen that we are going to use Retrofit for consuming Appnexus API, the following post I will show you how exactly it could be done. 

Kindly post a comment if you either like it or if you feel the content is inappropriate in some way.