Monday, July 30, 2012

What Scala?

After 2 years with Scala, our company (GLMX) decided to finally say goodbye to the language that was (or is still perhaps) the future of Java. We did the unthinkable of converting all the code back to Java and was pleasantly surprised that it wasn't as bloated, non-intuitive and verbose as we thought it would be. On the contrary, the code is what it is (what-you-code-is-what-you-get), no implicit conversions, inferencing, traits that modified behavior (that really should have been done with composition), coupled with the fact that there are nice Java libraries out there (Google Guava for one) that encapsulates a lot of the functional aspects that we liked about Scala (never mind the fact that functions are really classes in Scala), it's actually refreshing to know that we don't have to do a whole lot of ".seqAsJavaList", ".bigDecimal" and teach other frameworks (e.g. Spring) to understand what's a scala Int). It would have been nice if all frameworks were indeed written in Scala but alas, we live in a world where interoperability is a requirement, not a nice-to-have.

Couple of learnings about Scala (copied from a chat I had with an ex-Googler friend of mine asking about my thoughts about it):
  1. It's hard to read the code quickly (code concepts / 100 char line) with all the implicit conversions, language specific peculiarities, compositions, etc., you used to be able to read code by glancing, with scala you can't because the density of logic vs. characters can suddenly jump within a single line (e.g. one line you have an assignment to a val, the next one is the reversal of a list, implicitly converted from a java list, then a flatmap on the output that extracts a collection, partitioning by some predicate, ... all on one line)
  2. Build tools don't work perfectly with scala (some application servers don't like the code it generates, but that's not to blame the language itself), it doesn't co-exist with java in some corner cases (meaning we need to write java anyways, JPA for one, if you need typed criteria query meta-model classes)
  3. Hard to find scala programmers and training them takes a while (from java)
  4. The actual productivity gains are not that much after a while; an IDE is good for eliminating a lot of the cruft of java and you realize that you don't actually save that much from coding
  5. The libraries encourage you to trash the heap quite a bit without too much thought (since it's so easy), with java though, you are actually more aware of allocations and performance since there's less magic happening
Then the more specific issues:
  • XML support in scala is nice but when you need true marshalling and unmarshalling, you are back to some java-based framework
  • The provided types are nice but once you start mixing java frameworks and scala frameworks, you have to be real careful passing in a Scala list of BigDecimal since even if the conversion is done properly to yield a Java list, the BigDecimals in it still needs another round of unwrapping
  • It's actually nice to just use iteration rather than making it real "scala-ly" and issue a chain of function calls on collections to do what you want. I understand that you can still do iteration in Scala but it's just too tempting to be cool and instead instantiate a whole bunch of functions (which, again, are actual classes in the JVM) and create a bunch of throw-away intermediate collections
  • Then, there's always the debate about whether to pass a Buffer, List, Seq, Collection, Traversable, TraversableOnce around. :) With Java, it's easy to suggest that functions should take in the most flexible inputs while returning the most specific outputs (i.e. taking in Collection<T> and returning a List<T>)
Okay, I still really do like the fact that you don't need () everywhere and must end a line with a semi-colon though. That and the fact that you are more inclined to use "final" variables.

Friday, July 27, 2012

Finally, a working Nest Auto-Away with Google Latitude

This is a long time coming, I needed Indigo 5 (used as an event engine in this case), the Google Latitude Bridge I whipped up earlier (https://latitudebridge.appspot.com), the Nest for Indigo Plugin (https://github.com/johnray/Indigo-Nest-Thermostat-Plugin/blob/master/README.md) and my own Google Latitude for Indigo Script (http://www.perceptiveautomation.com/filelib/index.php?keywords=google+latitude).

Background: My Nest's Auto-Away never really worked properly since I live in a townhouse with 3 floors and I am the only person there. It never said that it detected enough activity and even when I tried to fool it (by purposefully "generating activity" in front of it when it's learning), it's still spotty at best. In any case, it will only turn on auto-away when no activity has been detected for 2 hours.

I would love to have it automatically detect when I am away accurately (saving me money) and I realized that everything is there technically: 1) an internet enabled thermostat, 2) a location tracking server and 3) a device that follows me all the time.

So, with a bit of tinkering, my Nest now has a couple new tricks: when I am away (>1km from home) for 5 minutes, it engages away status immediately; when I am back (<1km from home) for 5 minutes, it disengages away status. With Indigo, it actually says it out loud. :) When I am back in the house, it would say "away status for thermostat disengaged". Cool :)

Thursday, July 26, 2012

Hibernate Envers Bug HHH-7157

While helping out a co-worker today and looking at a stack trace, I got a deja-vu feeling that I have seen the bug before. Turns out it was the same manifestation where if you have a Many-to-One and One-to-Many relationship between two entities (one using embedeed ids) and they are audited, Hibernate generates a query that's not valid (talk about ORM errors), it would supply too many query parameters to the underlying database resulting in a nasty exception that you think typically don't exist anymore if you use an ORM framework.

Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Invalid value "7" for parameter "parameterIndex" [90008-143]


For those interested, I saw the bug back in March and filed a bug against Hibernate with a test case reproducing the bug on github: https://hibernate.onjira.com/browse/HHH-7157. Unfortunately, it's still not fixed in the latest release 4.1.5.SP1. The workaround is to add @NotAudited to the @OneToMany side of the relationship (which does not inhibit any auditing as the OneToMany side of the relationship isn't really represented in a database row anyways).

Full stack trace below (different DBMS will produce a different error message though since it's technically an error with the actual query):


javax.persistence.RollbackException: Error while committing the transaction
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:90)
at org.hibernate.clementp.EnversTest.testAuditedEmbeddedId(EnversTest.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:69)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:48)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:292)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:76)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:182)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Invalid value "7" for parameter "parameterIndex" [90008-143]
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1360)
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1288)
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:78)
... 31 more
Caused by: org.hibernate.exception.GenericJDBCException: Invalid value "7" for parameter "parameterIndex" [90008-143]
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
at org.hibernate.engine.jdbc.internal.proxy.AbstractStatementProxyHandler.continueInvocation(AbstractStatementProxyHandler.java:129)
at org.hibernate.engine.jdbc.internal.proxy.AbstractProxyHandler.invoke(AbstractProxyHandler.java:81)
at $Proxy22.setLong(Unknown Source)
at org.hibernate.type.descriptor.sql.BigIntTypeDescriptor$1.doBind(BigIntTypeDescriptor.java:57)
at org.hibernate.type.descriptor.sql.BasicBinder.bind(BasicBinder.java:92)
at org.hibernate.type.AbstractStandardBasicType.nullSafeSet(AbstractStandardBasicType.java:305)
at org.hibernate.type.AbstractStandardBasicType.nullSafeSet(AbstractStandardBasicType.java:300)
at org.hibernate.type.ComponentType.nullSafeSet(ComponentType.java:358)
at org.hibernate.persister.entity.AbstractEntityPersister.dehydrate(AbstractEntityPersister.java:2611)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2854)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3298)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:88)
at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:362)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:354)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:275)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:326)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:52)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1127)
at org.hibernate.envers.synchronization.AuditProcess.doBeforeTransactionCompletion(AuditProcess.java:157)
at org.hibernate.engine.spi.ActionQueue$BeforeTransactionCompletionProcessQueue.beforeTransactionCompletion(ActionQueue.java:662)
at org.hibernate.engine.spi.ActionQueue.beforeTransactionCompletion(ActionQueue.java:307)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:524)
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:105)
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:175)
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:73)
... 31 more
Caused by: org.h2.jdbc.JdbcSQLException: Invalid value "7" for parameter "parameterIndex" [90008-143]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
at org.h2.message.DbException.get(DbException.java:167)
at org.h2.message.DbException.getInvalidValueException(DbException.java:213)
at org.h2.jdbc.JdbcPreparedStatement.setParameter(JdbcPreparedStatement.java:1254)
at org.h2.jdbc.JdbcPreparedStatement.setLong(JdbcPreparedStatement.java:541)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.hibernate.engine.jdbc.internal.proxy.AbstractStatementProxyHandler.continueInvocation(AbstractStatementProxyHandler.java:122)
... 55 more

Google Latitude OAuth Bridge

As part of my project to use Indigo 5 (http://www.perceptiveautomation.com/indigo/index.html) to automate my house (and also remind me when I am forgetful of doing things like closing the garage door when I leave the house...), I decided to integrate Google Latitude into the server so that a variable "isAtHome" would be exposed and various triggers can decide on what to do (e.g. emailing me when nobody's at home but there's activity detected in the house).

Anyways, it turned out to be a bit of a difficult task since the Google Latitude API requires OAuth 2.0 with tokens that live for an hour. That wouldn't really work with something like Indigo which would have required me to store a whole bunch of stuff (client ids, client secrets, tokens, refresher tokens) and execute multiple round-trips against Google auth servers before you can get your location securely.

Well, decided that I might as well do what I do best and whipped up a Google App Engine + Play Framework + Siena + bootstrap combo that would allow me to generate a permanent URL for the JSON query.

After about 2 hours, http://latitudebridge.appspot.com is up. :)

Twitter's Bootstrap project is awesome!

Basically, you do the OAuth once and it generates a unique URL for you to query your current position. Just like the old days. :)

Next up, get the applescript written so that it can do a curl and figure out the rest.

First Post!

Finally decided to start a public blog on more technical stuff that would likely bore my Facebook friends but probably useful to others out there . :)