Home

log4j is now optional!

Starting with version 2.2.2, you can replace log4j with any logger that has an slf4j binding. For example, to use slf4j-simple, declare your dependencies as shown below.

Gradle projects

compile ("com.emc.ecs:object-client:3.1.3") {
    exclude group: 'org.slf4j', module: 'slf4j-log4j12'
}
runtime "org.slf4j:slf4j-simple:1.7.5"

Maven projects

<dependency>
  <groupId>com.emc.ecs</groupId>
  <artifactId>object-client</artifactId>
  <version>3.1.3</version>
  <scope>compile</scope>
  <exclusions>
    <exclusion>
      <artifactId>slf4j-log4j12</artifactId>
      <groupId>org.slf4j</groupId>
    </exclusion>
  </exclusions>
</dependency>
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-simple</artifactId>
  <version>1.7.5</version>
  <scope>runtime</scope>
</dependency>

Maven coordinates:

Group	Artifact	Version
com.emc.ecs	object-client	3.1.3

Usage

There are two ways to use the client: with or without the "Smart Client". If your system has a traditional load balancer set up behind a VIP, then use the URI constructor of S3Config. Note that virtual-host-style requests (where bucket.namespace is prepended to the host) are not enabled by default. To enable this, call setUseVHost(true).

If you want to use the software load balancer built-in to the client and have direct access to all the ECS nodes by IP, then use the alternate constructors of S3Config and the smart client will use the hosts you provide to discover all the ECS node IPs and load balance across them automatically.

S3Config config;

// with load balancer (and corresponding baseUrl/DNS setup)
config = new S3Config(new URI("https://s3.mycompany.com"));
// if you use virtual-host-style requests:
config.setUseVHost(true);

// client-side load balancing (direct to individual nodes)
// single VDC
config = new S3Config(Protocol.HTTP, NODE_IP1, NODE_IP2);
// multiple VDCs
config = new S3Config(Protocol.HTTP, new Vdc("Boston", VDC1_NODE1, VDC1_NODE2),
        new Vdc("Seattle", VDC2_NODE1, VDC2_NODE2));
// to enable geo-pinning (hashes the object key and pins it to a specific VDC)
config.setGeoPinningEnabled(true);

config.withIdentity(S3_ACCESS_KEY_ID).withSecretKey(S3_SECRET_KEY);

S3Client s3Client = new S3JerseyClient(config);

Sharing a Singleton

This client is thread-safe, so multiple threads can share a single instance of the client. This is highly recommended, since constructing and destroying instances are relatively expensive operations, and each instance consumes some amount of resources.

Closing

Be sure to close the client instance when you are done using it. There is a background thread associated with the smart-client, and potentially other resources that need to be cleaned up when the client shuts down. You can do this by explicitly calling destroy() when you are completely done using the client:

// close() method on some arbitrary component that owns an S3 client
public void close() {
    try {
        if (s3Client != null) s3Client.destroy();
    } catch (Exception e) {
        log.warn("could not destroy S3 client", e);
    }
    super.close();
}

---

HTTP Client Differences

In high throughput environments (10GbE+) and with certain versions of the JVM, the apache HTTP client (used by default) has been shown to be less efficient than the native JVM HTTP client (HttpURLConnection). To maximize performance in these situations, you can configure the S3 client to use HttpURLConnection instead by using the optional constructor

// you need to increase Java's connection pool limits if you have more than 5 threads
System.setProperty("http.maxConnections", "100");
S3Client s3Client = new S3JerseyClient(config, new URLConnectionClientHandler());

Note that this handler does not support Expect: 100-Continue behavior if that is important to you. However, we have witnessed up to a 5x throughput increase with a simultaneous .5x CPU load using this handler with certain older releases of Java 8 (we have not tested this with recent updates).

From version 4.0.0, the constructors to create a client have been changed.

// Default Apache client
S3JerseyClient(S3Config s3Config);
// All type of clients, like HttpURLConnection
S3JerseyClient(S3Config config, String clientTransportConnector);

which means, you need to specify the 2nd param if Apache is not your choice. For example, you can create an HttpURLConnection client by S3JerseyClient(config, "HTTPURLCONNECTION"). For now, we only support these two types, but stay tuned for more.