Archive

Archive for May, 2018

MaybeGZIPInputStream

May 29th, 2018 No comments

I’m currently working on an application that persists Java serialized data (using ObjectOutputStream) in a database. Java’s serialization format compresses very well – so why not compress the data when storing it then decompress it while reading for a quick win? The problem is that there will still be legacy, uncompressed data, which the application will not be able to access if it assumes all data is now gzipped.

The solution is to use MaybeGZIPInputStream instead of GZIPInputStream. For example, when reading, instead of:

ObjectInputStream ois = new ObjectInputStream(new GZIPInputStream(databaseInputStream));

use MaybeGZIPInputStream instead:

ObjectInputStream ois = new ObjectInputStream(new MaybeGZIPInputStream(databaseInputStream));

And always write data using GZIPOutputStream. Now all of that existing data can be still be read, and newly written data gets the benefit of taking up much less storage (and taking up far less bandwidth / time being transferred between the application servers and the database).

Here’s the source code of MaybeGZIPInputStream:

import java.io.IOException;
import java.io.InputStream;
import java.io.PushbackInputStream;
import java.util.zip.GZIPInputStream;

/** Detect if the given {@link InputStream} contains compressed data. If it does, wrap it in a {@link GZIPInputStream}. If it doesn’t, don’t.
* @author Craig Andrews
*
*/
public class MaybeGZIPInputStream extends InputStream {

private final InputStream in;

public MaybeGZIPInputStream(final InputStream in) throws IOException {
final PushbackInputStream pushbackInputStream = new PushbackInputStream(in, 2);
if(isGZIP(pushbackInputStream)) {
this.in = new GZIPInputStream(pushbackInputStream);
}else {
this.in = pushbackInputStream;
}
}

private boolean isGZIP(final PushbackInputStream pushbackInputStream) throws IOException {
final byte[] bytes = new byte[2];
final int bytesRead = pushbackInputStream.read(bytes);
if(bytesRead > 0) {
pushbackInputStream.unread(bytes, 0, bytesRead);
}
if(bytesRead == 2) {
if ((bytes[0] == (byte) (GZIPInputStream.GZIP_MAGIC)) && (bytes[1] == (byte) (GZIPInputStream.GZIP_MAGIC >> 8))){
return true;
}
}
return false;
}

public int read() throws IOException {
return in.read();
}

public int hashCode() {
return in.hashCode();
}

public int read(byte[] b) throws IOException {
return in.read(b);
}

public boolean equals(Object obj) {
return in.equals(obj);
}

public int read(byte[] b, int off, int len) throws IOException {
return in.read(b, off, len);
}

public long skip(long n) throws IOException {
return in.skip(n);
}

public String toString() {
return in.toString();
}

public int available() throws IOException {
return in.available();
}

public void close() throws IOException {
in.close();
}

public void mark(int readlimit) {
in.mark(readlimit);
}

public void reset() throws IOException {
in.reset();
}

public boolean markSupported() {
return in.markSupported();
}

}

Categories: Uncategorized Tags:

SQS JMS Resource Adapter

May 7th, 2018 No comments

The recently released SQS JMS Resource Adapter allows JEE applications (running on any JEE application server, including Glassfish, Payara, JBoss, IBM Liberty, etc) to easily use AWS SQS as a JMS implementation. This resource adapter can be helpful in many situations, such as:

  • Migrating an existing JEE application from another JMS implementation (such as RabbitMQ, ActiveMQ, IBM MQ, etc) to AWS SQS.
  • Allowing the JMS implementation to be switched out. For example, developers can use the ActiveMQ resource adapter, and in production, this AWS SQS resource adapter could be used.

Grab the resource adapter from Maven Central and submit issues and pull requests over at GitHub.

Categories: Uncategorized Tags:

Trusting DoD Certificates in Docker and Beanstalk

May 1st, 2018 No comments

The US DoD (Department of Defense) uses its own root certificate when signing https certificates for its domains. For example, https://www.my.af.mil/ uses such a certificate. These root certificates are not trusted by any (commercial/public) operating system, browser, or other client. Therefore, in order to access these sites and not get an error, the DoD certificates must be trusted.

On Windows, go to DISA’s PKI and PKE Tools page and under “Trust Store” follow the directions for the “InstallRoot X: NIPR Windows Installer”

On Linux, download the certificates from MilitaryCAC’s Linux Information page (direct link to the certificates). Then follow your distribution’s instructions on how to install certificates to the trust store. For example, on Red Hat / CentOS / Fedora / Amazon Linux, copy the certificates to /etc/pki/ca-trust/source/anchors/ then run update-ca-trust. On Debian / Ubuntu and Gentoo, copy the certificates to /usr/local/share/ca-certificates/ then run update-ca-certificates.

On Docker, for a Red Hat / CentOS / Fedora / Amazon Linux (or other Fedora-type system) derived container, add the following to the Dockerfile:

RUN yum -y install openssl \
&& CERT_BUNDLE="Certificates_PKCS7_v5.3_DoD" \
&& curl "https://iasecontent.disa.mil/pki-pke/${CERT_BUNDLE}.zip" --output certs.zip \
&& unzip certs.zip "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" \
&& openssl pkcs7 -in "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" -print_certs -out "/etc/pki/ca-trust/source/anchors/${CERT_BUNDLE}.pem" \
&& update-ca-trust \
&& update-ca-trust force-enable \
&& rm -rf certs.zip "${CERT_BUNDLE}" \
&& yum -y remove openssl \
&& rm -rf /var/cache/yum

On AWS Elastic Beanstalk the .ebextensions mechanism can be used. In the jar/war/etc deployment archive, add these files:
.ebextensions/install_dod_certificates.config
container_commands:
01install_dod_certificates:
command: "bash .ebextensions/scripts/install_dod_certificates.sh"

.ebextensions/scripts/install_dod_certificates.sh
#!/bin/bash
set -e # stop on all errors
yum install -y unzip openssl
CERT_BUNDLE="Certificates_PKCS7_v5.3_DoD"
curl "https://iasecontent.disa.mil/pki-pke/${CERT_BUNDLE}.zip" --output certs.zip
unzip certs.zip "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b"
openssl pkcs7 -in "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" -print_certs -out "/etc/pki/ca-trust/source/anchors/${CERT_BUNDLE}.pem"
update-ca-trust
update-ca-trust force-enable
rm -rf certs.zip "${CERT_BUNDLE}"
yum -y remove unzip
rm -rf /var/cache/yum

Categories: Uncategorized Tags: