Archive

Archive for the ‘Uncategorized’ Category

SQS JMS Resource Adapter

May 7th, 2018 No comments

The recently released SQS JMS Resource Adapter allows JEE applications (running on any JEE application server, including Glassfish, Payara, JBoss, IBM Liberty, etc) to easily use AWS SQS as a JMS implementation. This resource adapter can be helpful in many situations, such as:

  • Migrating an existing JEE application from another JMS implementation (such as RabbitMQ, ActiveMQ, IBM MQ, etc) to AWS SQS.
  • Allowing the JMS implementation to be switched out. For example, developers can use the ActiveMQ resource adapter, and in production, this AWS SQS resource adapter could be used.

Grab the resource adapter from Maven Central and submit issues and pull requests over at GitHub.

Categories: Uncategorized Tags:

Trusting DoD Certificates in Docker and Beanstalk

May 1st, 2018 No comments

The US DoD (Department of Defense) uses its own root certificate when signing https certificates for its domains. For example, https://www.my.af.mil/ uses such a certificate. These root certificates are not trusted by any (commercial/public) operating system, browser, or other client. Therefore, in order to access these sites and not get an error, the DoD certificates must be trusted.

On Windows, go to DISA’s PKI and PKE Tools page and under “Trust Store” follow the directions for the “InstallRoot X: NIPR Windows Installer”

On Linux, download the certificates from MilitaryCAC’s Linux Information page (direct link to the certificates). Then follow your distribution’s instructions on how to install certificates to the trust store. For example, on Red Hat / CentOS / Fedora / Amazon Linux, copy the certificates to /etc/pki/ca-trust/source/anchors/ then run update-ca-trust. On Debian / Ubuntu and Gentoo, copy the certificates to /usr/local/share/ca-certificates/ then run update-ca-certificates.

On Docker, for a Red Hat / CentOS / Fedora / Amazon Linux (or other Fedora-type system) derived container, add the following to the Dockerfile:

RUN yum -y install openssl \
&& CERT_BUNDLE="Certificates_PKCS7_v5.3_DoD" \
&& curl "https://iasecontent.disa.mil/pki-pke/${CERT_BUNDLE}.zip" --output certs.zip \
&& unzip certs.zip "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" \
&& openssl pkcs7 -in "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" -print_certs -out "/etc/pki/ca-trust/source/anchors/${CERT_BUNDLE}.pem" \
&& update-ca-trust \
&& update-ca-trust force-enable \
&& rm -rf certs.zip "${CERT_BUNDLE}" \
&& yum -y remove openssl \
&& rm -rf /var/cache/yum

On AWS Elastic Beanstalk the .ebextensions mechanism can be used. In the jar/war/etc deployment archive, add these files:
.ebextensions/install_dod_certificates.config
container_commands:
01install_dod_certificates:
command: "bash .ebextensions/scripts/install_dod_certificates.sh"

.ebextensions/scripts/install_dod_certificates.sh
#!/bin/bash
set -e # stop on all errors
yum install -y unzip openssl
CERT_BUNDLE="Certificates_PKCS7_v5.3_DoD"
curl "https://iasecontent.disa.mil/pki-pke/${CERT_BUNDLE}.zip" --output certs.zip
unzip certs.zip "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b"
openssl pkcs7 -in "${CERT_BUNDLE}/${CERT_BUNDLE}.pem.p7b" -print_certs -out "/etc/pki/ca-trust/source/anchors/${CERT_BUNDLE}.pem"
update-ca-trust
update-ca-trust force-enable
rm -rf certs.zip "${CERT_BUNDLE}"
yum -y remove unzip
rm -rf /var/cache/yum

Categories: Uncategorized Tags:

Coal to Cryptocurrency: Mining Remains a Threat

November 30th, 2017 No comments

Coal was the fuel that powered the Industrial Revolution, bootstrapping the modern age as we know it. Acquiring it was simple, using it was easy, and it got the job done. Coal was the perfect resource. Back in those days, efficiency and cleanliness weren’t concerns because of ecological immaturity (society didn’t know any better) and scale (industry wasn’t big enough to impact the world sufficiently to raise concerns).

Cryptocurrency mining is today’s coal mining, and it’s time to start considering alternative solutions.

With any currency (traditional or cryptographic), a few constraints must be in place: a unit of currency cannot be spent more than once (no “double spending”), transactions must complete in a timely manner, and everyone must agree when a transaction is complete. With traditional paper money, it’s clear how all of these constraints are satisfied: counterfeiting is made difficult by secure notes and strongly discouraged by legal means, the transaction completes when physical possession of the note is transferred, and all parties can look at their physical possession of notes to determine a transaction’s state.

Implementing these constraints digitally is more difficult than when using physical items. Bitcoin, being the world’s first cryptocurrency, used the best solutions available. The system bitcoin leverages is known as blockchain with “proof of work.” Bitcoin uses a series of blocks, appropriately referred to as a blockchain, that forms a ledger which records the state of every bitcoin since bitcoin’s inception. Each block records the movement of a number of bitcoins between owners: the proof. In order for a block to be valid, it can include each coin at most once (to prevent double spending); it must include the unique identity (the hash) of the previous block; and it must include the solution to a difficult math problem (a cryptographic hash). The process of solving these problems to form valid blocks is known as mining and those who do so are called miners.

Solving these mining challenges takes hardware, infrastructure, cooling, and the electricity to keep it all going. To incentivize the block discovery process, the system rewards the miner with a predetermined amount of currency. To satisfy the need for timely transactions, each includes a transaction fee to be awarded to the miner. Therefore a miner wants to include as many transactions as possible into a block in order to collect the greatest amount in fees. Once a block has been mined, it’s shared to the public so anyone can verify that there was no double spending, and that the cryptographic hash is valid. Miners only mine new blocks on top of valid ones.

As cryptocurrencies grow more valuable, the mining rewards grow as well, making mining increasingly lucrative. This  draws in more miners which, in turn, uses more energy. As of November 2017, each bitcoin transaction now uses as much energy as the average American house consumes in a week. Furthermore, as a country, bitcoin now ranks as the 69th highest energy consumer.

Just as coal was a great way to bootstrap industry, proof of work has done a great job bootstrapping cryptocurrencies. But neither coal nor proof of work are viable paths forward; they’re simply too polluting. So what are the solar panel and wind turbine analogues for cryptocurrency?

One system is proof of stake. At a high level, this system limits miners’ output in proportion to the total amount of currency the miner owns. For example, if there are 200 units of currency total and a miner owns 10 units, that miner may only contribute 5% of the mining power. In this way, there’s no race for miners to acquire massive computational resources. This system has other advantages over proof of work as well including avoiding the 51% attack problem. Ethereum, the second largest cryptocurrency by market capitalization, is currently in the process of switching from proof of work to proof of stake. Ark, Dash, and Neo are examples of cryptocurrencies currently using a proof of stake system.

Another system is known as “the tangle,” currently only used by the IOTA cryptocurrency. The tangle’s alternative methodology provides many advantages over blockchain, including zero transaction fees, no miner energy expenditure, and greater decentralization. However, analogous to alternative energy sources in days past, the tangle today is not as proven, researched, tested, or understood as well as blockchain systems are.

In this modern age of global climate change, the world needs to abandon proof of work systems. With their energy expenditures exceeding that of most countries, the cost to the environment is simply too great to continue down this path, especially since alternatives already exist. Like modern industry’s move away from familiar, reliable coal, it’s time for the cryptocurrency community to move on from proof of work to better, more responsible solutions.

Categories: Uncategorized Tags:

Log4jdbc Spring Boot Starter

March 27th, 2017 No comments

Logging SQL as it’s executed is a fairly common desire when developing applications. Perhaps an ORM (such as Hibernate) is being used, and you want to see the actual SQL being executed. Or maybe you’re tracking down a performance problem and need to know if it’s in the application or the database, so step #1 is finding out what query is executing and for how long.

Solving this problem once and for all (at least for Spring Boot applications), I created Log4jdbc Spring Boot Starter. It’s a very simple yet powerful way to log SQL queries (and more, such as timing information). And unlike other solution, the queries logged are ready to run – the ‘?’ parameters are replaced with their values. This means you can copy and paste the query from the log and run them unmodified in the SQL query tool of your choice, saving a lot of time.

For background, my motivation for this work is a result of a Spring Boot / Hibernate application I have in progress. I started by using spring.jpa.properties.hibernate.show_sql=true but that only logs queries with ‘?’ place holders. To log the values, add spring.jpa.properties.hibernate.type=trace. At least now I had the query and the values for it, but to run it in my query tool (I need to EXPLAIN the query), I had to replace each ‘?’ with the value – and I had over 20 place holders. That got old fast.

There are other approaches to log queries, such as the one described in Display SQL to Console in Spring JdbcTemplate. I’m not a fan of this approach because it only works for queries made through JdbcTemplate (so Hibernate queries aren’t logged, for example) and it’s an awfully lot of code to include and therefore have to maintain in each project.

I discovered Log4jdbc but it’s a bit of a pain to setup in a Spring Boot application because it:

  • doesn’t use the Spring Environment (application.properties)
  • needs setup to wrap the DataSource’s in the Log4jdbc DataSourceSpy

Wanting to solve this problem precisely once and never again, I created Log4jdbc Spring Boot Starter.

To use it, just add to your project:

  1. <dependency>
  2.   <groupId>com.integralblue</groupId>
  3.   <artifactId>log4jdbc-spring-boot-starter</artifactId>
  4.   <version>[INSERT VERSION HERE]</version>
  5. </dependency>

Then turn on the logging levels as desired in application.properties, for example:


logging.level.jdbc.sqlonly=DEBUG

When no logging is configured (all loggers are set to fatal or off), log4jdbc returns the original Connection.

See the Log4jdbc Spring Boot Starter project page for more information.

Categories: Uncategorized Tags:

Working around HHH-9663: Orphan removal does not work for OneToOne relations

March 23rd, 2017 No comments

HHH-9663 means that orphan removal doesn’t work for OneToOne relationships. For example, given File and FileContent as below (taken from the bug report):

  1. package pl.comit.orm.model;
  2.  
  3. import javax.persistence.Entity;
  4. import javax.persistence.FetchType;
  5. import javax.persistence.Id;
  6. import javax.persistence.OneToOne;
  7.  
  8. @Entity
  9. public class File {
  10.  
  11. 	private int id;
  12.  
  13. 	private FileContent content;
  14.  
  15. 	@Id
  16. 	public int getId() {
  17. 		return id;
  18. 	}
  19.  
  20. 	public void setId(int id) {
  21. 		this.id = id;
  22. 	}
  23.  
  24. 	@OneToOne(fetch = FetchType.LAZY, orphanRemoval = true)
  25. 	public FileContent getContent() {
  26. 		return content;
  27. 	}
  28.  
  29. 	public void setContent(FileContent content) {
  30. 		this.content = content;
  31. 	}
  32. }
  1. package pl.comit.orm.model;
  2.  
  3. import javax.persistence.Entity;
  4. import javax.persistence.Id;
  5.  
  6. @Entity
  7. public class FileContent {
  8.  
  9. 	private int id;
  10.  
  11. 	@Id
  12. 	public int getId() {
  13. 		return id;
  14. 	}
  15.  
  16. 	public void setId(int id) {
  17. 		this.id = id;
  18. 	}
  19. }
  1. package pl.comit.orm.dao;
  2.  
  3. import javax.persistence.EntityManager;
  4. import javax.persistence.PersistenceContext;
  5.  
  6. import org.springframework.stereotype.Repository;
  7. import org.springframework.transaction.annotation.Transactional;
  8.  
  9. import pl.comit.orm.model.File;
  10. import pl.comit.orm.model.FileContent;
  11.  
  12. @Repository
  13. public class Dao {
  14.  
  15. 	@PersistenceContext
  16. 	private EntityManager entityManager;
  17.  
  18. 	@Transactional
  19. 	public void assureCreatedTaskAndNote(int fileId, int contentId) {
  20. 		FileContent content = entityManager.find(FileContent.class, contentId);
  21. 		if (content == null) {
  22. 			content = new FileContent();
  23. 			content.setId(contentId);
  24. 			entityManager.persist(content);
  25. 		}
  26.  
  27. 		File file = entityManager.find(File.class, fileId);
  28. 		if (file == null) {
  29. 			file = new File();
  30. 			file.setId(fileId);
  31. 			entityManager.persist(file);
  32. 		}
  33. 		file.setContent(content);
  34. 	}
  35.  
  36. 	@Transactional
  37. 	public void removeContent(int fileId) {
  38. 		File file = entityManager.find(File.class, fileId);
  39. 		file.setContent(null);
  40. 	}
  41.  
  42. 	public FileContent find(int contentId) {
  43. 		return entityManager.find(FileContent.class, contentId);
  44. 	}
  45. }

Running this as the main class will result in an exception:

  1. package pl.comit.orm;
  2.  
  3. import org.springframework.context.support.ClassPathXmlApplicationContext;
  4.  
  5. import pl.comit.orm.dao.Dao;
  6. import pl.comit.orm.model.FileContent;
  7.  
  8. public final class Application {
  9.  
  10. 	private static final String CFG_FILE = "applicationContext.xml";
  11.  
  12. 	public static void main(String[] args) {
  13. 		test(new ClassPathXmlApplicationContext(CFG_FILE).getBean(Dao.class));
  14. 	}
  15.  
  16. 	public static void test(Dao dao) {
  17. 		dao.assureCreatedTaskAndNote(1, 2);
  18. 		dao.removeContent(1);
  19. 		FileContent content = dao.find(2);
  20. 		if (content != null) {
  21. 			System.err.println("Content found: " + content);
  22. 		}
  23. 	}
  24. }

A workaround is to manually remove and detach the old referent, and then persist the new referent. Here’s an updated File.java:

  1. package pl.comit.orm.model;
  2.  
  3. import org.springframework.stereotype.Component;
  4.  
  5. import javax.annotation.PostConstruct;
  6. import javax.annotation.PreDestroy;
  7. import javax.persistence.Entity;
  8. import javax.persistence.FetchType;
  9. import javax.persistence.Id;
  10. import javax.persistence.OneToOne;
  11. import javax.persistence.PersistenceContext;
  12.  
  13. @Entity
  14. public class File {
  15.  
  16. 	private int id;
  17.  
  18. 	private FileContent content;
  19.  
  20. 	@Id
  21. 	public int getId() {
  22. 		return id;
  23. 	}
  24.  
  25. 	public void setId(int id) {
  26. 		this.id = id;
  27. 	}
  28.  
  29. 	@OneToOne(fetch = FetchType.LAZY, orphanRemoval = true)
  30. 	public FileContent getContent() {
  31. 		return content;
  32. 	}
  33.  
  34. 	public void setContent(FileContent content) {
  35. 		if(this.content != content){
  36. 			final oldContent = this.content;
  37. 			this.content = content;
  38. 			if(oldContent!=null){
  39. 				// Hibernate won't remove the oldContent for us, so do it manually; workaround HHH-9663
  40. 				WorkaroundHHH9663.entityManager.remove(oldContent);
  41. 				WorkaroundHHH9663.entityManager.detach(oldContent);
  42. 			}
  43. 		}
  44. 	}
  45.  
  46. 	// WORKAROUND https://hibernate.atlassian.net/browse/HHH-9663 "Orphan removal does not work for OneToOne relations"
  47. 	@Component
  48. 	public static class WorkaroundHHH9663 {
  49. 		@PersistenceContext
  50. 		private EntityManager injectedEntityManager;
  51.  
  52. 		private static EntityManager entityManager;
  53.  
  54. 		@PostConstruct
  55. 		public void postConstruct(){
  56. 			entityManager = injectedEntityManager;
  57. 		}
  58.  
  59. 		@PreDestroy
  60. 		public void preDestroy(){
  61. 			this.entityManager = null; // NOPMD
  62. 		}
  63. 	}
  64. 	// END WORKAROUND
  65. }

Note that no Dao changes were made, so if, for example, Spring Data was used instead of such a Dao, you wouldn’t have to modify anything else. And you can just remove this workaround code easily when a version of Hibernate because available with HHH-9963 fixed.
Finally, yes, this approach does exhibit a bit of code smell (the use of the static variable in this way and the entity having a container managed component aren’t exactly best practices), but, it’s a workaround – hopefully just a temporary one.

Categories: Uncategorized Tags:

Spring Boot, HTTPS required, and Elastic Beanstalk health checks

March 9th, 2017 No comments

Spring Boot can be very easily configured to require HTTPS for all requests. In application.properties, simply set

security.require-ssl=true

And that works great – until you’re running the Spring Boot application on AWS Elastic Beanstalk with both HTTP and HTTPS listeners:

In that case, Elastic Beanstalk’s health check is always done over HTTP. The configuration page even says as much (there is no option to change it to be over HTTPS):

Since Spring Boot will redirect all non-secure HTTP requests to HTTPS, the health check will see an HTTP 302 redirect and therefore fail.

To workaround this issue (and in my opinion, AWS shortcoming), Spring Boot needs to be configured to allow insecure requests to the health check URL. To do so, we’ll define a new URL (/aws-health) that proxies Actuator’s health URL but responds over http and https.

In your security configuration class (named WebSecurityConfiguration below as an example) which extends WebSecurityConfigurerAdapter, add the following to the existing implementation of configure(HttpSecurity) (or create that method if it doesn’t already exist):

  1. import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
  2.  
  3. import org.springframework.security.config.annotation.web.builders.HttpSecurity;
  4. import org.springframework.security.web.access.channel.ChannelDecisionManagerImpl;
  5.  
  6. public class WebSecurityConfiguration extends WebSecurityConfigurerAdapter {
  7.   @Override
  8.   protected void configure(final HttpSecurity http) throws Exception {
  9.     // Elastic Beanstalk health checks only happen over HTTP, so as a workaround
  10.     // create a new URL (/aws-health) that forwards to the Actuator health check, see InsecureHealthController
  11.     // That URL is set to respond over any channel (not just secure, aka https, ones)
  12.     // see https://candrews.integralblue.com/2017/03/spring-boot-https-required-and-elastic-beanstalk-health-checks/
  13.     http.requiresChannel()
  14.       .antMatchers("/aws-health").requires(ChannelDecisionManagerImpl.ANY_CHANNEL)
  15.       .anyRequest().requiresSecure();
  16.   }
  17. }

Now create the controller:

  1. import java.io.IOException;
  2.  
  3. import javax.servlet.ServletException;
  4. import javax.servlet.http.HttpServletRequest;
  5. import javax.servlet.http.HttpServletRequestWrapper;
  6. import javax.servlet.http.HttpServletResponse;
  7.  
  8. import org.springframework.boot.actuate.autoconfigure.ManagementServerProperties;
  9. import org.springframework.stereotype.Controller;
  10. import org.springframework.web.bind.annotation.RequestMapping;
  11. import org.springframework.web.bind.annotation.RequestMethod;
  12. import org.springframework.web.util.UriComponentsBuilder;
  13.  
  14. /**
  15.  * Elastic Beanstalk health checks only happen over HTTP, so as a workaround
  16.  * create a new URL (/aws-health) that forwards to the Actuator health check
  17.  * That URL is set to respond over any channel (not just secure, aka https, ones) in {@link WebSecurityConfiguration}
  18.  * 
  19.  * @see https://candrews.integralblue.com/2017/03/spring-boot-https-required-and-elastic-beanstalk-health-checks/
  20.  */
  21. public class InsecureHealthController {
  22.   private final ManagementServerProperties management;
  23.  
  24.   public InsecureHealthController(ManagementServerProperties management) {
  25.     this.management = management;
  26.   }
  27.  
  28.   @RequestMapping(value="/health", method=RequestMethod.GET)
  29.   public void health(final HttpServletRequest request, final HttpServletResponse response) throws ServletException, IOException{
  30.     final String healthUrl = UriComponentsBuilder.fromPath(management.getContextPath()).path("/aws-health").toUriString();
  31.     final HttpServletRequestWrapper requestWrapper = new HttpServletRequestWrapper(request){
  32.       @Override
  33.         public boolean isSecure() {
  34.           return true;
  35.         }
  36.       };
  37.       request.getRequestDispatcher(healthUrl).forward(requestWrapper, response);
  38.     }
  39. }
Categories: Uncategorized Tags:

Spring Cache Abstraction as a Hibernate Cache Provider

February 23rd, 2017 No comments

Many of the projects I’ve worked on over my career have leveraged a Spring/Hibernate stack. Over that time, the Spring/Hibernate integration has greatly improved making the once tedious and repetitive chore of setting up a new project (and maintaining an existing one through upgrades, expansion, and refactoring) far simpler. Now it’s as simple as going to the Spring Initializr and selecting the right options. In seconds, you get a nicely configured Spring Boot boilerplate and you’re ready for the real work to begin.

Well, most of the setup is simple at least… setting up the Hibernate cache is still not simple. To get decent performance on Hibernate, the query cache and/or second level caches are key. But they’re a pain to configure and that configuration is almost entirely redundant with the Spring cache configuration.

Spring has a cache abstraction that can back to ehcache, redis, memcache, caffeine, guava, and a variety of other implementations – including just a simple ConcurrentHashMap (which is great for test harnesses). Spring Boot makes it really easy to setup. Instead of setting up the Spring cache and the Hibernate cache totally separately, why not use the same configuration and same implementations for both?

That’s what Hibernate SpringCache does. Set up Spring Cache, add the Hibernate SpringCache dependency to your Spring Boot project, tell Hibernate to enable the query cache and/or second level cache, and you’re done.

Hopefully, Hibernate itself will one day include this functionality. If you’d like to see Hibernate/Spring integration become that much simpler, let your thoughts be known by commenting on the pull request.

Categories: Uncategorized Tags:

Spring ID to Entity Conversion

July 22nd, 2015 1 comment

When using Spring with Hibernate or JPA, it can be very convenient to use objects as method parameters and automatically convert primary keys to those objects. For example:

  1. @RequestMapping(value = "/person/{personId}", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
  2. @ResponseBody
  3. public PersonDto getById(@PathVariable Person person) {
  4. 	return person;
  5. }

instead of:

  1. @RequestMapping(value = "/person/{personId}", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
  2. @ResponseBody
  3. public PersonDto getById(@PathVariable String personId) {
  4. 	return personService.getById(personId);
  5. }

Using this approach throughout a project (in controllers, services, etc) can lead to much more readable, shorter code.

Spring Data JPA provides a similar feature via DomainClassConverter. In my case, Spring Data wasn’t an option.

Here’s how to make this happen. First, set up a @Configuration class:

  1. package com.integralblue.candrews.config;
  2.  
  3. import javax.annotation.PostConstruct;
  4.  
  5. import org.springframework.context.annotation.Bean;
  6. import org.springframework.context.annotation.Configuration;
  7. import org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport;
  8.  
  9. import com.integralblue.candrews.converter.EntityToIdConverter;
  10. import com.integralblue.candrews.converter.IdToEntityConverter;
  11.  
  12. @Configuration
  13. public class WebMvcConfig extends WebMvcConfigurationSupport {
  14. 	@Bean
  15. 	protected IdToEntityConverter idToEntityConverter() {
  16. 		return new IdToEntityConverter();
  17. 	}
  18.  
  19. 	@Bean
  20. 	protected EntityToIdConverter entityToIdConverter() {
  21. 		return new EntityToIdConverter();
  22. 	}
  23.  
  24. 	@PostConstruct
  25. 	public void addConverters() {
  26. 		mvcConversionService().addConverter(idToEntityConverter());
  27. 		mvcConversionService().addConverter(entityToIdConverter());
  28. 	}
  29. }

Now add these 2 classes:

  1. package com.integralblue.candrews.converter;
  2.  
  3. import java.util.Collections;
  4. import java.util.HashSet;
  5. import java.util.Set;
  6.  
  7. import javax.persistence.EntityManager;
  8. import javax.persistence.PersistenceContext;
  9. import javax.persistence.metamodel.Attribute;
  10. import javax.persistence.metamodel.EntityType;
  11. import javax.persistence.metamodel.SingularAttribute;
  12.  
  13. import org.springframework.beans.factory.annotation.Autowired;
  14. import org.springframework.core.convert.TypeDescriptor;
  15. import org.springframework.core.convert.converter.ConditionalGenericConverter;
  16. import org.springframework.format.support.FormattingConversionService;
  17.  
  18. /**
  19.  * Converts an entity to its ID (the field in the entity annotated with
  20.  * {@link javax.persistence.Id}). This is similar to
  21.  * {@link org.springframework.data.repository.support.DomainClassConverter<T>}
  22.  * but since we don't use Spring Data, we can't use that.
  23.  */
  24. public class EntityToIdConverter implements ConditionalGenericConverter {
  25.  
  26. 	@PersistenceContext
  27. 	private EntityManager entityManager;
  28.  
  29. 	@Autowired
  30. 	private FormattingConversionService conversionService;
  31.  
  32. 	@Override
  33. 	public Set<ConvertiblePair> getConvertibleTypes() {
  34. 		return Collections.singleton(new ConvertiblePair(Object.class,
  35. 				Object.class));
  36. 	}
  37.  
  38. 	@SuppressWarnings("unchecked")
  39. 	@Override
  40. 	public boolean matches(TypeDescriptor sourceType, TypeDescriptor targetType) {
  41. 		EntityType<?> entityType;
  42. 		try {
  43. 			entityType = entityManager.getMetamodel().entity(
  44. 					sourceType.getType());
  45. 		} catch (IllegalArgumentException e) {
  46. 			return false;
  47. 		}
  48. 		if (entityType == null)
  49. 			return false;
  50.  
  51. 		// In my opinion, this is probably a bug in Hibernate.
  52. 		// If the class has a @Id that is a complex type (such as another
  53. 		// entity), then entityType.getIdType() will throw an
  54. 		// IllegalStateException with the message "No supertype found"
  55. 		// To work around this, get the idClassAttributes, and if there is
  56. 		// exactly 1, use the java type of that.
  57. 		Class<?> idClass;
  58. 		if (entityType.hasSingleIdAttribute()) {
  59. 			idClass = entityType.getIdType().getJavaType();
  60. 		} else {
  61. 			Set<SingularAttribute<?, ?>> idAttributes = new HashSet<>();
  62. 			@SuppressWarnings("rawtypes")
  63. 			Set attributes = entityType.getAttributes();
  64. 			for (Attribute<?, ?> attribute : (Set<Attribute<?, ?>>) attributes) {
  65. 				if (attribute instanceof SingularAttribute<?, ?>) {
  66. 					SingularAttribute<?, ?> singularAttribute = (SingularAttribute<?, ?>) attribute;
  67. 					if (singularAttribute.isId()) {
  68. 						idAttributes.add(singularAttribute);
  69. 					}
  70. 				}
  71.  
  72. 			}
  73. 			if (idAttributes.size() == 1) {
  74. 				idClass = idAttributes.iterator().next().getJavaType();
  75. 			} else {
  76. 				return false;
  77. 			}
  78. 		}
  79.  
  80. 		return conversionService.canConvert(TypeDescriptor.valueOf(idClass),
  81. 				targetType);
  82. 	}
  83.  
  84. 	@Override
  85. 	public Object convert(Object source, TypeDescriptor sourceType,
  86. 			TypeDescriptor targetType) {
  87. 		if (sourceType.getType() == targetType.getType()) {
  88. 			return source;
  89. 		}
  90. 		if (source == null) {
  91. 			return null;
  92. 		}
  93. 		EntityType<?> entityType = entityManager.getMetamodel().entity(
  94. 				sourceType.getType());
  95. 		Object ret = entityManager.getEntityManagerFactory()
  96. 				.getPersistenceUnitUtil().getIdentifier(source);
  97. 		if (ret == null)
  98. 			return null;
  99. 		return conversionService.convert(ret,
  100. 				TypeDescriptor.valueOf(entityType.getIdType().getJavaType()),
  101. 				targetType);
  102. 	}
  103. }
  1. package com.integralblue.candrews.converter;
  2.  
  3. import java.util.Collections;
  4. import java.util.HashSet;
  5. import java.util.Set;
  6.  
  7. import javax.persistence.EntityManager;
  8. import javax.persistence.NoResultException;
  9. import javax.persistence.PersistenceContext;
  10. import javax.persistence.metamodel.Attribute;
  11. import javax.persistence.metamodel.EntityType;
  12. import javax.persistence.metamodel.SingularAttribute;
  13.  
  14. import org.apache.commons.lang.StringUtils;
  15. import org.springframework.beans.factory.annotation.Autowired;
  16. import org.springframework.core.convert.TypeDescriptor;
  17. import org.springframework.core.convert.converter.ConditionalGenericConverter;
  18. import org.springframework.format.support.FormattingConversionService;
  19. import org.springframework.security.access.prepost.PostAuthorize;
  20.  
  21. /**
  22.  * Converts an ID (the field in the entity annotated with
  23.  * {@link javax.persistence.Id}) to the entity itself. This is similar to
  24.  * {@link org.springframework.data.repository.support.DomainClassConverter<T>}
  25.  * but since we don't use Spring Data, we can't use that.
  26.  */
  27. public class IdToEntityConverter implements ConditionalGenericConverter {
  28.  
  29. 	@PersistenceContext
  30. 	private EntityManager entityManager;
  31.  
  32. 	@Autowired
  33. 	private FormattingConversionService conversionService;
  34.  
  35. 	@Override
  36. 	public Set<ConvertiblePair> getConvertibleTypes() {
  37. 		return Collections.singleton(new ConvertiblePair(Object.class,
  38. 				Object.class));
  39. 	}
  40.  
  41. 	@SuppressWarnings("unchecked")
  42. 	@Override
  43. 	public boolean matches(TypeDescriptor sourceType, TypeDescriptor targetType) {
  44. 		EntityType<?> entityType;
  45. 		try {
  46. 			entityType = entityManager.getMetamodel().entity(
  47. 					targetType.getType());
  48. 		} catch (IllegalArgumentException e) {
  49. 			return false;
  50. 		}
  51. 		if (entityType == null)
  52. 			return false;
  53.  
  54. 		// In my opinion, this is probably a bug in Hibernate.
  55. 		// If the class has a @Id that is a complex type (such as another
  56. 		// entity), then entityType.getIdType() will throw an
  57. 		// IllegalStateException with the message "No supertype found"
  58. 		// To work around this, get the idClassAttributes, and if there is
  59. 		// exactly 1, use the java type of that.
  60. 		Class<?> idClass;
  61. 		if (entityType.hasSingleIdAttribute()) {
  62. 			idClass = entityType.getIdType().getJavaType();
  63. 		} else {
  64. 			Set<SingularAttribute<?, ?>> idAttributes = new HashSet<>();
  65. 			@SuppressWarnings("rawtypes")
  66. 			Set attributes = entityType.getAttributes();
  67. 			for (Attribute<?, ?> attribute : (Set<Attribute<?, ?>>) attributes) {
  68. 				if (attribute instanceof SingularAttribute<?, ?>) {
  69. 					SingularAttribute<?, ?> singularAttribute = (SingularAttribute<?, ?>) attribute;
  70. 					if (singularAttribute.isId()) {
  71. 						idAttributes.add(singularAttribute);
  72. 					}
  73. 				}
  74.  
  75. 			}
  76. 			if (idAttributes.size() == 1) {
  77. 				idClass = idAttributes.iterator().next().getJavaType();
  78. 			} else {
  79. 				return false;
  80. 			}
  81. 		}
  82.  
  83. 		return conversionService.canConvert(sourceType,
  84. 				TypeDescriptor.valueOf(idClass));
  85. 	}
  86.  
  87. 	@Override
  88. 	@PostAuthorize("hasPermission(returnObject, 'view')")
  89. 	public Object convert(Object source, TypeDescriptor sourceType,
  90. 			TypeDescriptor targetType) {
  91. 		if (sourceType.getType() == targetType.getType()) {
  92. 			return source;
  93. 		}
  94. 		EntityType<?> entityType = entityManager.getMetamodel().entity(
  95. 				targetType.getType());
  96. 		Object id = conversionService.convert(source, sourceType,
  97. 				TypeDescriptor.valueOf(entityType.getIdType().getJavaType()));
  98. 		if (id == null
  99. 				|| (id instanceof String && StringUtils.isEmpty((String) id))) {
  100. 			return null;
  101. 		}
  102. 		Object ret = entityManager.find(targetType.getType(), id);
  103. 		if (ret == null) {
  104. 			throw new NoResultException(targetType.getType().getName()
  105. 					+ " with id '" + id + "' not found");
  106. 		}
  107. 		return ret;
  108. 	}
  109. }
Categories: Uncategorized Tags:

Issues using Ehcache with ARC as the Hibernate Cache

July 22nd, 2015 No comments

When using Ehcache’s ARC (Automatic Resource Control) on a cache acting as the Hibernate cache, these kinds of warning will likely appear:

WARN [pool-1-thread-1] [ehcache.pool.impl.DefaultSizeOfEngine] sizeOf The configured limit of 100 object references was reached while attempting to calculate the size of the object graph. This can be avoided by adding stop points with @IgnoreSizeOf annotations. Since the CacheManger or Cache elements maxDepthExceededBehavior is set to “abort”, the sizing operation has stopped and the reported cache size is not accurate. If performance degradation is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache elements maxDepth attribute. For more information, see the Ehcache configuration documentation.

To fix this problem, you need to tell ARC what parts of Hibernate objects shouldn’t be counted when calculating object sizes so ARC doesn’t get into an infinite loop (when it hits circular references) or try to calculate the size of the entire Hibernate object graph.

One way to solve this problem is to use ehcache-sizeofengine-hibernate. To do so, simply add the jar to your application’s class path and it will be automatically discovered and used by Ehcache. However, since ehcache-sizeofengine-hibernate isn’t in Maven Central, or any publicly accessible Maven repository, it’s not really useful. I requested that it be published to Central a while ago, but the project seems to be unmaintained – so a more maintainable way to solve this problem is required.

A system property needs to be set pointing to a file that lists the classes and fields that should be ignored by ARC. In my Spring @Configuation class for configuring ehcache, I have this block:

static {
System.setProperty(net.sf.ehcache.pool.impl.DefaultSizeOfEngine.USER_FILTER_RESOURCE, "net.sf.ehcache.sizeof.filter.properties");
}

But you can really do this anywhere (including simply adding “-Dnet.sf.ehcache.sizeof.filter=net.sf.ehcache.sizeof.filter.properties” to the Java command line).

Next, create a file named “net.sf.ehcache.sizeof.filter.properties” in the classpath (in a Maven project, the src/main/resources directory makes sense) with these contents:

org.hibernate.Session
org.hibernate.internal.SessionImpl
org.hibernate.engine.spi.sessionimplementor
org.hibernate.cache.spi.QueryKey.positionalParameterTypes
org.hibernate.proxy.AbstractLazyInitializer.entityName
org.hibernate.proxy.pojo.BasicLazyInitializer.persistentClass
org.hibernate.proxy.pojo.BasicLazyInitializer.getIdentifierMethod
org.hibernate.proxy.pojo.BasicLazyInitializer.setIdentifierMethod
org.hibernate.proxy.pojo.BasicLazyInitializer.componentIdType
org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer
org.hibernate.engine.spi.TypedValue.type
Categories: Uncategorized Tags:

Issues using Ehcache with ARC as the Thymeleaf Cache

July 22nd, 2015 No comments

Ehcache’s ARC (Automatic Resource Control) is pretty great – it more or less automatically and optimally handles cache sizing/allocation. Using ARC, one can effectively say, “Here’s 2GB to allocate for caching – no more than that – manage the cache as best as possible.” Of course, more complex rules can also be done (on a per cache basis instead of globally, for instance) but that’s the general idea.

I’ve used ARC on my last couple of projects to great success. However, it’s not without a bit of complexity. ARC has to calculate the size of objects in the cache to determine how much space is used by the cache, and calculating sizes of Java objects isn’t easy or obvious. Besides difficult to do yet seemingly simple sound tasks as answering “How big is an object reference pointer?” there are object graph traversal problems, and those object graph issues are the ones that I’ve run into a few times in my ARC adventures.

When using Ehcache with ARC to store Thymeleaf templates (for example, when using Spring Cache abstraction backed by Ehcache as the Thymeleaf cache implementation as I previous discussed), you’ll run into messages like this one:

WARN [pool-1-thread-1] [ehcache.pool.impl.DefaultSizeOfEngine] sizeOf The configured limit of 100 object references was reached while attempting to calculate the size of the object graph. This can be avoided by adding stop points with @IgnoreSizeOf annotations. Since the CacheManger or Cache elements maxDepthExceededBehavior is set to “abort”, the sizing operation has stopped and the reported cache size is not accurate. If performance degradation is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache elements maxDepth attribute. For more information, see the Ehcache configuration documentation.

To fix this problem, you need to tell ARC what parts of Thymeleaf objects shouldn’t be counted when calculating object sizes so ARC doesn’t get into an infinite loop (when it hits circular references) or try to calculate the size of the entire Spring context.

A system property needs to be set pointing to a file that lists the classes and fields that should be ignored by ARC. In my Spring @Configuation class for configuring ehcache, I have this block:

static {
System.setProperty(net.sf.ehcache.pool.impl.DefaultSizeOfEngine.USER_FILTER_RESOURCE, "net.sf.ehcache.sizeof.filter.properties");
}

But you can really do this anywhere (including simply adding “-Dnet.sf.ehcache.sizeof.filter=net.sf.ehcache.sizeof.filter.properties” to the Java command line).

Next, create a file named “net.sf.ehcache.sizeof.filter.properties” in the classpath (in a Maven project, the src/main/resources directory makes sense) with these contents:

org.thymeleaf.templateresolver.TemplateResolution.resourceResolver
org.thymeleaf.dom.Node.processors

That’s it!

Note that I’ve requested that Thymeleaf make the org.thymeleaf.Template class serializable, which would remove the need to do anything (ARC would “Just Work” if this change is made): https://github.com/thymeleaf/thymeleaf/issues/378 So hopefully, with a future version of Thymeleaf, all of this information will be useless and nothing more than of historical interest.

Categories: Uncategorized Tags: