Just like the year before, I was in the fortunate position of being able to attend Google I/O. When I was still living in Virginia, such a trip would have been only wishful thinking, so I am very grateful for such opportunities. The meetings were amazing, and smarter and more eloquent people than myself have already reported more than enough about it. I will try not to bore anyone by yet another rehash of the Wave demo, or all the other cool things that have been announced. In other words, I will keep this short ;-)
One of the talks I attended was called Writing Real-Time Games for Android, and it turned out one of my favourite sessions. I will not pretend to have understood everything (Chris Pruett certainly lost me when he started talking about the different graphics features on the device), but it was inspiring to see the quality of output that could be achieved on such a device. And it was all in Java, a language that I actually understood.
When I first blogged about my new hobby-project of making the JOGRE gaming engine work with an App Engine backend, I got asked why I was going after a framework that used a Java client, when Javascript and HTML canvas seemed the new way to go. My response was relatively bland. Bascially, I did not really care about building a state-of-the-art client; I more cared about porting an existing backend. Now after this talk I start wondering: could I achieve both? JOGRE is Java -- Android is Java!
JOGRE builds clients on a regular Java framework with a Swing or AWT based frontend. I am already working on making that client work "on the web", by replacing socket-based messaging with an http protocol. Beyond that, how hard would it be to refactor the client to work on Android backends, rather than regular PCs? Would it be possible to "mobilize" the games with little effort? Imagine having a standard gaming platform for multiplayer games, backed by App Engine and running on your cell phone? How easy would that make writing new online games, and pushing them to the Android marketplace?
So... I guess I've got to ask: any "Android fan" reading this blog who could audit the client side of JOGRE and give me some feedback?
Sunday, May 31, 2009
Thursday, May 21, 2009
An initial refactoring
This is the first technical post in my new JOGRE series . As mentioned in the previous article, the goal behind this project is to explore how App Engine can efficiently host online games that scale to many users. Since I do not want to reinvent the wheel, I am basing the work on an existing Java-based platform (http://jogre.sourceforge.net/main.htm ) that already happens to come with a lot of prewritten games. How hard is it going to be to retrofit that base to the App Engine platform? Over the next couple of weeks, I will hopefully find out.
The post is a little bit longer than usual, so I'll do my best to put in a few meaningful headlines inbetween. This way, readers can skip to the parts that are of particular interest to them.
A lot of my activities over the next couple of weeks will be about refactoring code in JOGRE, so I'd like to start out by summarizing the guidelines that I will try to follow in this work. Note that the "you" in the following paragraphs does not address the reader but myself. I know -- talking to myself is a bad habit, but it makes it easier to write the rules down. I will to try my best to abide by them in this and other refactoring posts.
The cardinal rule: Don't be a jerk
Life's too short to get angry, so why would anyone want to work with someone he or she does not like? This is even more important to consider when there is no transfer of money involved: people work on this kind of software because they have a passion for it and it is fun. So, try to avoid spoiling the project for anyone else!
You might have an extensive background in methodology xyz. You might have read the GOF book
, Martin Fowler's Refactoring
, and you might have quoted from Effective Java
on occasion. You may think you know how to spot "code smells", or that you have read and written a ton of code. Good for you, but do yourself a favor: forget all of this right now! Unless you are a glutton for punishment, you have chosen to work on this project in your spare time because you like it, it works well 90% of the time, and you'd like to make it even better. This means that if you run into something that does not comply with what you might consider quasi-standards of good coding, it is probably like that for one of the following reasons:
It's funny 'cause it's true: nobody should be looking at your changes and ask, "why the heck did he do that?!?" There are several things that can significantly increase the WTFs per minute ratio:
Your open source work is branched off an existing project (Note: I contacted Bob Marks before I started any of this. We agreed to keep my work separate for now, but if I am successful and he finds the changes useful and beneficial, we will merge them back into the main project). The overall amount of changes you are going to make will most likely affect not even ten percent of the overall codebase. The new features you add will probably be even fewer. Therefore, do not imply that you "built" this platform, or forget to give credit where credit is due. If you build a new feature, feel free to put your name into the author tag -- but if all you are doing is pulling code from one class into an other class, that doesn't make you the author. Give credit where credit is due! Also, do not expect that your changes will definitely make it back verbatim into the main branch (in other words, don't get defensive if the other coders require some additional modifications first). Do not remove any branding that the original authors might have put in and replace it with your own -- it might be ok to do that in customized deploys, but not in the open source project itself. That being said -- of course you are free to take some liberties with the project, as long as you comply with its open source license (GPL v2 or higher in case of JOGRE). Just don't forget about the cardinal rule.
Sometimes, as you get deeper into the code base, you might discover that the project does not really do what you need it to do. It might be designed in a way that you cannot easily adapt, there could be fundamental disagreements between you and the other team members -- or you simply happen to run out of time and need to focus on something else. It happens on occasion, so move on if you need to move on. However, if you choose to do so, communicate this clearly and in a nice manner. Do not promise to implement a particular feature and just never do it. Do not write a scathing goodbye post that explains why project xyz is so much better. Sometimes, the real problem actually lies somewhere between the keyboard and the seat.
Simply put, the goal of my initial work is to rework any code I might find in JOGRE that would prevent it from running on App Engine. Since I am just starting out in this effort, I do not have a very good idea yet what that might be. However, I am aware of a few things that work in a regular Java program but would be challenging in App Engine, such as:
After having successfully built the server and played a game of Connect4 with myself, I started looking into the main method of the server code. From the documentation, I knew that the client-server protocol was xml based, which is good. I did not know however in which way those messages were exchanged. Many gaming engines use TCP directly (or, for better throughput, even UDP), since it provides a lot of nice properties (like establishing an keeping up a connection) that are more than sufficient for their needs.
I took a peek and found my expectations confirmed: JOGRE was using a well-established pattern that could have been straight from The Java Tutorial
(see http://java.sun.com/docs/books/tutorial/networking/sockets/clientServer.html ):
ServerConnectionThread is a subclass of AbstractConnectionThread, which contains all the logic of how to fetch data from the connection, maintain the lifecycle of the socket, update connection state, and remember the name of the current user that owns the connection. AbstractConnectionThread, a subclass of java.lang.Thread, also had a couple of other subclasses that shared the connection handling logic but did other things with the arriving data. How could I start squeezing http support in here without breaking anything?
There is a very good chapter in Working Effectively with Legacy Code
that introduces a concept that is called the Single-Responsibility Principle:
The refactoring itself was pretty straightforward: I took the content of the thread's run()-method, plus everything that used the socket object, and copied it into the new class. The thread's constructor would simply wrap the original Socket into the new object, and it's start/stop/run-methods would delegate calls to their equivalents in the new object.
I compiled the code and ran the unit tests -- everything still worked :-). The next step was to simply kick out those delegating, hollowed-out methods. I added a "getMessageBus" method to the base class and had all subclasses use that to get access to the new MessageBus object, and called the moved methods in the subclasses:
Now that the the SocketBasedMessageBus was doing all the socket-based work, AbstractConnectionThread no longer needed to subclass the Thread class. This enabled me to get rid of the run() method, and encapsulate the use of Threads in the message bus:
While looking at the new class, I realized that the names of the methods I had moved into SocketBasedMessageBus were focused around threading and loops -- but that was not necessarily what the class was responsible for. As the name said, SocketBasedMessageBus isolated the exchange of messages via a Java socket -- so its public API should reflect that. I decided to rename the methods accordingly:
While probably still not single responsibility, I now had a situation where the AbstractConnectionThread class no longer had any particular knowledge that it was using a socket -- except for its constructor. That was good enough for me (after all, if it ain't broke...), so I decided to wrap it up. One final cleanup step remained: the constructor of our base class should not have to know about sockets. Nor would our MessageBus really have to care about the user name of the connection thread, as long as there was a parse- and a cleanup method. I therefore chose to extract those aspects of the class into interfaces and have the base classes depend on those rather than the concrete implementations:
At the end of the day, the refactoring of AbstractConnectionThread resulted in the following modification:
For those amongst us who'd rather just read code, here are the classes affected by this refactoring:
The original class
http://code.google.com/p/gae-ogre/source/browse/trunk/api/src/org/jogre/common/AbstractConnectionThread.java?r=2
Main refactoring targets
Affected classes (minor changes)
The post is a little bit longer than usual, so I'll do my best to put in a few meaningful headlines inbetween. This way, readers can skip to the parts that are of particular interest to them.
A hobbyist's code of conduct to refactoring
A lot of my activities over the next couple of weeks will be about refactoring code in JOGRE, so I'd like to start out by summarizing the guidelines that I will try to follow in this work. Note that the "you" in the following paragraphs does not address the reader but myself. I know -- talking to myself is a bad habit, but it makes it easier to write the rules down. I will to try my best to abide by them in this and other refactoring posts.
The cardinal rule: Don't be a jerk
Life's too short to get angry, so why would anyone want to work with someone he or she does not like? This is even more important to consider when there is no transfer of money involved: people work on this kind of software because they have a passion for it and it is fun. So, try to avoid spoiling the project for anyone else!
Corollary #1: If it ain't broke, don't fix it.
You might have an extensive background in methodology xyz. You might have read the GOF book
, Martin Fowler's Refactoring, and you might have quoted from Effective Java on occasion. You may think you know how to spot "code smells", or that you have read and written a ton of code. Good for you, but do yourself a favor: forget all of this right now! Unless you are a glutton for punishment, you have chosen to work on this project in your spare time because you like it, it works well 90% of the time, and you'd like to make it even better. This means that if you run into something that does not comply with what you might consider quasi-standards of good coding, it is probably like that for one of the following reasons:- The code has evolved historically towards the way it currently is, and it has never been too much of a problem.
- The project team has a different philosophy or architectural view towards development than you. Happens all the time, and that doesn't make it bad code. Feel free to ask the team why it is the way it is, but do not expect them to change it for you.
- You ran into a bug or an area with "TODOs". If it's an issue and the team agrees, feel free to improve it -- as long as you do it with the consent of the others.
Corollary #2: Honor the code's spirit and document the intentions of changes you make.
It's funny 'cause it's true: nobody should be looking at your changes and ask, "why the heck did he do that?!?" There are several things that can significantly increase the WTFs per minute ratio:
- changes that would make sense -- if the author had only documented why he or she is doing them
- too many concurrent modifications in one iteration
- combining a refactoring with writing new features
- incomplete or speculative refactorings (in other words, starting a change because "it might come in handy later" but never following through on that)
- major changes to the contract of a core class that require code changes throughout the project
- random changes just for the heck of it, or because one does not like the way a particular piece of code looks (see corollary #1). While it sometimes makes sense to clean up something small while you're in a particular class anyway, there is a fine line between cleanup and major modifications.
Corollary #3: Contributorship does not imply ownership
Your open source work is branched off an existing project (Note: I contacted Bob Marks before I started any of this. We agreed to keep my work separate for now, but if I am successful and he finds the changes useful and beneficial, we will merge them back into the main project). The overall amount of changes you are going to make will most likely affect not even ten percent of the overall codebase. The new features you add will probably be even fewer. Therefore, do not imply that you "built" this platform, or forget to give credit where credit is due. If you build a new feature, feel free to put your name into the author tag -- but if all you are doing is pulling code from one class into an other class, that doesn't make you the author. Give credit where credit is due! Also, do not expect that your changes will definitely make it back verbatim into the main branch (in other words, don't get defensive if the other coders require some additional modifications first). Do not remove any branding that the original authors might have put in and replace it with your own -- it might be ok to do that in customized deploys, but not in the open source project itself. That being said -- of course you are free to take some liberties with the project, as long as you comply with its open source license (GPL v2 or higher in case of JOGRE). Just don't forget about the cardinal rule.
Corollary #4: Failure IS an option
Sometimes, as you get deeper into the code base, you might discover that the project does not really do what you need it to do. It might be designed in a way that you cannot easily adapt, there could be fundamental disagreements between you and the other team members -- or you simply happen to run out of time and need to focus on something else. It happens on occasion, so move on if you need to move on. However, if you choose to do so, communicate this clearly and in a nice manner. Do not promise to implement a particular feature and just never do it. Do not write a scathing goodbye post that explains why project xyz is so much better. Sometimes, the real problem actually lies somewhere between the keyboard and the seat.
The overall goal of my refactorings
Simply put, the goal of my initial work is to rework any code I might find in JOGRE that would prevent it from running on App Engine. Since I am just starting out in this effort, I do not have a very good idea yet what that might be. However, I am aware of a few things that work in a regular Java program but would be challenging in App Engine, such as:
- spawning threads
- network communication through anything but http
- storing data in files
- keeping data in a static field and expecting it to stick around (the next request might hit a completely different instance of my App Engine app)
- connecting via JDBC to a database
- longer living background processes
- anything that requires a "restart" of the system
- anything equivalent to a "global scan" (will the client ever need to get the list of all users?)
- anything with near-realtime requirements that relies on the accuracy of timestamps
- anything that uses native code or expects certain operating system commands to be available
- code that uses any class that may not be whitelisted
Today's refactoring in pictures
After having successfully built the server and played a game of Connect4 with myself, I started looking into the main method of the server code. From the documentation, I knew that the client-server protocol was xml based, which is good. I did not know however in which way those messages were exchanged. Many gaming engines use TCP directly (or, for better throughput, even UDP), since it provides a lot of nice properties (like establishing an keeping up a connection) that are more than sufficient for their needs.
I took a peek and found my expectations confirmed: JOGRE was using a well-established pattern that could have been straight from The Java Tutorial
(see http://java.sun.com/docs/- a main class (JogreServer) creates a ServerSocket and listens in a loop to incoming connections
- for each incoming connection, it starts a new Thread (ServerConnectionThread) that then handles all the gaming logic.
ServerConnectionThread is a subclass of AbstractConnectionThread, which contains all the logic of how to fetch data from the connection, maintain the lifecycle of the socket, update connection state, and remember the name of the current user that owns the connection. AbstractConnectionThread, a subclass of java.lang.Thread, also had a couple of other subclasses that shared the connection handling logic but did other things with the arriving data. How could I start squeezing http support in here without breaking anything?
There is a very good chapter in Working Effectively with Legacy Code
that introduces a concept that is called the Single-Responsibility Principle:"Every class should have a single responsibility: It should have a single purpose in the system, and there should be only one reason to change it."Using that as the foundation of my first refactoring, I decided to break parts of the AbstractConnectionThread out into a new class called SocketBasedMessageBus:
The refactoring itself was pretty straightforward: I took the content of the thread's run()-method, plus everything that used the socket object, and copied it into the new class. The thread's constructor would simply wrap the original Socket into the new object, and it's start/stop/run-methods would delegate calls to their equivalents in the new object.
I compiled the code and ran the unit tests -- everything still worked :-). The next step was to simply kick out those delegating, hollowed-out methods. I added a "getMessageBus" method to the base class and had all subclasses use that to get access to the new MessageBus object, and called the moved methods in the subclasses:
Now that the the SocketBasedMessageBus was doing all the socket-based work, AbstractConnectionThread no longer needed to subclass the Thread class. This enabled me to get rid of the run() method, and encapsulate the use of Threads in the message bus:
While looking at the new class, I realized that the names of the methods I had moved into SocketBasedMessageBus were focused around threading and loops -- but that was not necessarily what the class was responsible for. As the name said, SocketBasedMessageBus isolated the exchange of messages via a Java socket -- so its public API should reflect that. I decided to rename the methods accordingly:
While probably still not single responsibility, I now had a situation where the AbstractConnectionThread class no longer had any particular knowledge that it was using a socket -- except for its constructor. That was good enough for me (after all, if it ain't broke...), so I decided to wrap it up. One final cleanup step remained: the constructor of our base class should not have to know about sockets. Nor would our MessageBus really have to care about the user name of the connection thread, as long as there was a parse- and a cleanup method. I therefore chose to extract those aspects of the class into interfaces and have the base classes depend on those rather than the concrete implementations:
At the end of the day, the refactoring of AbstractConnectionThread resulted in the following modification:
- instead of managing sockets and threads, an AbstractConnectionThread is connected to a generic MessageBus object, which may choose any transport protocol (sockets, udp, http) it likes. The class no longer has any dependencies on sockets.
- AbstractConnectionThread has two remaining responsibilities: to manage the name of the user that the connection belongs to, and to provide a generalization for how to react to incoming messages from the MessageBus. The latter responsibility is represented by the MessageParser interface, which is what the MessageBus interfaces with.
- concrete subclasses like ServerConnectionThread still accept Sockets in their constructor (thus remaining compatible with the rest of the codebase), but they wrap the socket in a MessageBus before pushing it into the base constructor. I might choose to refactor that in a later stage, but if I do so, I can do it on an individual class basis, without breaking any of the other peer classes.
The refactoring in code
For those amongst us who'd rather just read code, here are the classes affected by this refactoring:
The original class
http://code.google.com/p/gae-
Main refactoring targets
Affected classes (minor changes)
Sunday, May 17, 2009
My summer project: let the games begin
I finally got my Schluesselmeister app to a point where I feel comfortable moving away from my desktop app and into the cloud (the final missing piece was sharing, so that my wife and I could use the same key database). Now that that's done, I need another hobby project :-) The search criteria were as following
After some more soul searching, I decided to focus on something else instead: online gaming. A recent lightning talk I saw on Buddypoke inspired me: obviously, App Engine is a great backend to scale fun applications to millions of users. But how? What works, and what doesn't? What are best practices to build such a fun and massively scalable application?
Naturally, I did not want to reinvent the wheel. I searched a little bit around and found JOGRE, a Java-based gaming engine that is open source, seems to have decent test coverage and comes with a ton of pre-implemented games. How hard would it be to run this backend on App Engine? I am going to find out over the next months (yes, months -- I have no idea how hard it is going to be; and the weather is way to beautiful outside to be coding all weekend ;-). I am starting with the 0.3 source snapshot, which I uploaded to http://code.google.com/p/gae-ogre/. I doubt I will have anything runnable for quite a while, but I hope the journey of getting there will yield some interesting posts. Wish me luck :-)
- It should be as far away from my day job as it could possibly be, yet still involve Google App Engine.
- It should force me to look at the tools from a different angle and broaden my perspective.
- It should be about something I can "occasionally" mention in this blog and that people will hopefully enjoy reading.
- It should be something that I find fun to do.
Even if Google happens to stay out of the market, its products will help lower the entry-level for this industry even more. By using its products, a new breed of systems, based on Internet technologies and open standards will reach the market. Greater competition will improve the overall quality of the solutions offered, and the end user is going to benefit.I was briefly contemplating putting that statement to the test and writing such an application. After all, App Engine seemed like the perfect backend of moving such software into the cloud. The problem with it: it seemed more like work than fun. Also, I wasn't sure if I could squeeze any good articles out of such a project ;-)
After some more soul searching, I decided to focus on something else instead: online gaming. A recent lightning talk I saw on Buddypoke inspired me: obviously, App Engine is a great backend to scale fun applications to millions of users. But how? What works, and what doesn't? What are best practices to build such a fun and massively scalable application?
Naturally, I did not want to reinvent the wheel. I searched a little bit around and found JOGRE, a Java-based gaming engine that is open source, seems to have decent test coverage and comes with a ton of pre-implemented games. How hard would it be to run this backend on App Engine? I am going to find out over the next months (yes, months -- I have no idea how hard it is going to be; and the weather is way to beautiful outside to be coding all weekend ;-). I am starting with the 0.3 source snapshot, which I uploaded to http://code.google.com/p/gae-ogre/. I doubt I will have anything runnable for quite a while, but I hope the journey of getting there will yield some interesting posts. Wish me luck :-)
Thursday, May 14, 2009
Notes from the last meetup
[updated 3/18/09: added reference to webvnc]
Yesterday, I was at the last developers meetup in Palo Alto. As always, it was a very interesting time, as people gave lightning talks about many different subjects. Check out this link in a couple of days from now, as the organizer is going to link in more links and details from different presenters.
A couple of things that I found particularly interesting (in the order in which they were presented):
Yesterday, I was at the last developers meetup in Palo Alto. As always, it was a very interesting time, as people gave lightning talks about many different subjects. Check out this link in a couple of days from now, as the organizer is going to link in more links and details from different presenters.
A couple of things that I found particularly interesting (in the order in which they were presented):
- A member of the team behind BuddyPoke shared some lessons learned behind his secret to scaling to millions of users. Unfortunately, I do not remember them all, but two of them were the use of sharding techniques for things like counters and to avoid using queries wherever possible. According to the presenter, BuddyPoke gets its data from the store almost exclusively through loading by primary key, and they avoid putting indexes on data wherever possible (<shameless-plug>if you'd like to use similar techniques in Java, check out my previous posts on a simple, key-based datastore api and efficient global counters. You can also download the sourcecode directly from this open source project.</shameless-plug>).
- Another developer is working on jiql, a JDBC driver for App Engine. In the long term, projects like this could be essential to ease of porting sql-based applications to App Engine, so I think it's worth checking out. I'd love to hear especially if anyone is using it to porting things like ActiveRecord in Rails or php-based stuff...
- There was a very nice demo on patching django-based applications to use the Jinja2 templating system, including a real life example on how it simplified life in a web application that the developer was building.
- A demo of a prototype that can broadcast browser content to a set of viewers (useful for broadcasting demos and training sessions): http://www.webnc.net
Monday, May 11, 2009
Let the sunshine in
What a great weekend it was in the valley. The sky was clear, the sun was shining -- definitely not the time to sit at home and code! As summer finally starts making the outdoors fun again, I expect my output on this blog dropping accordingly. Therefore, no new code snippets or rants this week. Just a couple of links to keep you entertained.
- Rest in peace, Duke Nukem -- you have been one of my favorite college pastimes. As sad as it makes me, it shows however the importance of time to market: if your software project takes longer to develop than the atomic bomb, you might be in trouble. The Duke Nukem forever list has this and more examples of what has been accomplished in the same period of time.
- Here's a link to a recent talk called "what killed smalltalk" and the rant that made me aware of it ;-) Speaking of talks there is a thread on reddit where people recommend their favorite Google Tech Talks.
- If this has been way too serious for you, check out this Brief, Incomplete, and Mostly Wrong History of Programming Languages. My favorite fake history fact:
Lambdas are relegated to relative obscurity until Java makes them popular by not having them.
Wednesday, May 6, 2009
JDO and unit tests
Building App Engine apps with Java and GWT has become relatively easy, thanks to the great tool support with the eclipse plugin. Building these applications in a way that everything is easily testable is a little bit more work. The good news: once one has formed a habit of coding with tests in mind and learned a couple patterns on how to do it, it easily becomes second nature and produces leaner, cleaner, and more reliable software. On this blog, I occasionally post articles that focus a little more on that aspect of building App Engine apps (I'm starting to label them from now on for easier discoverability). This article is part of that series. Any code quoted in this post can be downloaded as open source and used under Apache license in any project. I would like to thank Max Ross, who provided a lot of inspiration in the unit tests for datanucleus-appengine and was also very helpful with advice as I was running into a couple of walls along the way :-)
One of the many things I liked about the Java App Engine launch is that right from the start, the team published Information on how to set up an App Engine based unit test. The recipe in those tests worked very well for me, especially since I was mostly working with the lower level DatastoreService and MemcacheService, not the standards based equivalents. Recently however, I wanted to test a class that uses JDO, so I needed a
Assume we have a class
For the
So far, so good -- but what about JDO? As it turns out,
Now that our
Assumed we have a simple data class similar to the Employee class. The following unit test is a simple example that simulates storing and loading objects from the store using JDO:
As shown above, once the
One of the many things I liked about the Java App Engine launch is that right from the start, the team published Information on how to set up an App Engine based unit test. The recipe in those tests worked very well for me, especially since I was mostly working with the lower level DatastoreService and MemcacheService, not the standards based equivalents. Recently however, I wanted to test a class that uses JDO, so I needed a
PersistenceManagerFactory. Getting such an object is not quite as straightforward: as documented, only one factory object per entire App Engine application should ever exist. If I try to load the factory more than once (like, say, once for each unit test), the system will actually throw an exception. Also, do I always need a jdoconfig.xml in my classpath? Turns out that the solution is not as complex as I thought at first. It took me a couple of attempts to get the setup right though, so I thought I'd share the recipe on my blog. As mentioned before, if you want to use these classes straight out of the box, you can also download them as a jar file.Assume we have a class
TestEnvironment, just like in the official instructions. With this class, we can create a straightforward TestInitializer class that can prepare an App Engine environment for us:public class TestInitializer {
private final ApiProxy.Environment environment;
private PersistenceManagerFactory pmf;
/**
* Constructor
*
* @param environmentOrNull
* the environment that should be used for the
* test (null to use the default)
*/
public TestInitializer(
ApiProxy.Environment environmentOrNull) {
this.environment =
(environmentOrNull == null) ? new TestEnvironment()
: environmentOrNull;
}
/** Constructor with default parameters */
public TestInitializer() {
this(null);
}
/**
* Sets up a unit test with the options stored in this
* object. This method should only be called once per
* object.
*/
public void setUp() throws Exception {
ApiProxyLocalImpl proxy =
new ApiProxyLocalImpl(new File(".")) {
};
proxy.setProperty(
LocalDatastoreService.NO_STORAGE_PROPERTY,
Boolean.TRUE.toString());
ApiProxy.setDelegate(proxy);
ApiProxy.setEnvironmentForCurrentThread(environment);
}
}
For the
tearDown (cleanup after a unit test), I found a useful trick in the JDOTestCase class. Successful tests are expected to clean up after themselves and not leave any open transactions behind, which is verified by the following code: public void tearDown(boolean testWasSuccessful)
throws Exception {
Transaction txn =
DatastoreServiceFactory.getDatastoreService()
.getCurrentTransaction(null);
try {
if (txn != null) {
try {
txn.rollback();
} finally {
if (testWasSuccessful) {
throw new IllegalStateException(
"Datastore service still has an active txn. Please "
+ "rollback or commit all txns before test completes.");
}
}
}
} finally {
ApiProxy.clearEnvironmentForCurrentThread();
}
}
So far, so good -- but what about JDO? As it turns out,
javax.jdo.JDOHelper has a very useful utility method. It constructs a new PersistenceManagerFactory from a Properties object instead of the jdoconfig.xml This might be highly inefficient in production, but for unit tests, it completely bypasses the create-only-one-factory-ever rule :-). The following tool method makes use of this trick: private PersistenceManagerFactory pmf;
public PersistenceManagerFactory getPersistenceManagerFactory() {
if (pmf == null) {
Properties newProperties = new Properties();
newProperties
.put(
"javax.jdo.PersistenceManagerFactoryClass",
"org.datanucleus.store.appengine.jdo."
+ DatastoreJDOPersistenceManagerFactory");
newProperties.put("javax.jdo.option.ConnectionURL",
"appengine");
newProperties.put(
"javax.jdo.option.NontransactionalRead", "true");
newProperties.put(
"javax.jdo.option.NontransactionalWrite", "true");
newProperties.put("javax.jdo.option.RetainValues",
"true");
newProperties.put(
"datanucleus.appengine.autoCreateDatastoreTxns",
"true");
newProperties.put(
"datanucleus.appengine.autoCreateDatastoreTxns",
"true");
pmf =
JDOHelper
.getPersistenceManagerFactory(newProperties);
}
return pmf;
}
Now that our
TestInitializer is complete, it is easy to wrap it in a unit test. You can look at the entire code here, so suffice it to say that I create an abstract BaseTest that uses the initializer above and also provides a convenience method that gets me a new PersistenceManager object: public PersistenceManager newPersistenceManager() {
return initializer.getPersistenceManagerFactory()
.getPersistenceManager();
}
Assumed we have a simple data class similar to the Employee class. The following unit test is a simple example that simulates storing and loading objects from the store using JDO:
package com.appenginefan.toolkit.unittests;
import java.util.Date;
import javax.jdo.PersistenceManager;
public class SimpleJDOTest
extends BaseTest {
public void testSetAndGet() {
// Create a new employee and write the object to the
// store
final Date date = new Date();
EmployeeData employee =
new EmployeeData("John", "Doe", date);
PersistenceManager manager = newPersistenceManager();
final Long id =
manager.makePersistent(employee).getId();
manager.close();
// From a different persistence manager, look up the
// object
manager = newPersistenceManager();
employee =
manager.getObjectById(EmployeeData.class, id);
assertEquals(employee.getFirstName(), "John");
assertEquals(employee.getLastName(), "Doe");
manager.close();
}
}
As shown above, once the
BaseTest class is written that does all the setup one needs (and which can be reused throughout the projects), adding new persistence related tests is no longer that cumbersome. And when it's easy to write good tests, it usually is also more fun -- which means, more tests get written and more bugs detected before code goes into production.
Friday, May 1, 2009
Finishing touches, part 2: wrapping it up
Welcome to the final part of iteration 4 on our little GWT based App Engine application. The deployed application can be reached at http://passwords.appenginefan.com -- give it a try and let me know what you think.
Since this is probably going to be the last post in the Schlüsselmeister series, I'd like to start with a quick overview of what has been covered so far. The original goal had been to kick the tires of the newly releases Java version of Google's App Engine. I wanted to taste the Kool Aid and build an application using the newly released tools (GWT + Integrated Eclipse tool). Overall it has been a great experience and I surely learned a lot. Things that were covered in previous posts included:
It was a lot of fun -- until today, where I faced my ultimate enemy: Cascading Style Sheets! It is sad, but I think I could not come up with a good looking page on my own if my life depended on it. I needed something to get started with. I needed a template!
A few google searches later, I found this link on mycelly.com. I'm certainly not an artist or expert in usability, but I know what I like when I see it. The site seemed simple (not too many flashy colors), clutter-free, and the designer explicitely allowed people to use the stuff for one's own page (I don't want to rip anyone off after all!). It even had a little extra area at the top that I could use to add a tipjoy button ;-).
With the foundation for my site found, I adapted the JSP to fit the mold:
I added the template to my stylesheet and fire up the application in Eclipse. So far, so good, but the program itself was still pretty bland! My solution to the problem is probably not the greates in the world, and if you have better ideas, please post them to the blog. What I did in this situation was: I opened up firebug.
Firebug is an awesome browser plugin that serves as a Javascript debugger, dom explorer and much more. One of the many cool things it does is its inspect functionality: it lets you select any section of a web page and shows its origin in the dom tree. I had just used GWT for the first time, and I had no idea (and no motivation to learn) how the web toolkit actually renders the UI Widgets into physical html. With Firebug, I could simply click and see what kind of element I had (was it a DIV? an INPUT? a SPAN?). Even better, the tool showed me exactly what the current style was, where the element "got" its style from (which css rules apply in which order? which are overwritten?). Last but not least, it also allowed me to change the style locally in the browser, which made it easy to try out how certain ideas look.
Back to styling my app. The first thing I did not like were how my dialogs looked. They did not have any borders, and their background was transparent (whatever text was behind the dialog would shine through). That had to be fixed! Fortunately, I had given both dialogs a style name in my GWT code, so I could easily address them in the dom tree:
Next stop was the split panel. I wanted the splitter itself and the left hand side (the list of categories) to be in the same color, so that it would look like one coherent unit. The color should be a gray tone, since it seemed to fit well into the overall theme of the template. Unlike the previous example, I had not really set any style names in the code, so I had to do a little digging using Firebug:
I also realized that the list of categories was placed just a little bit too high (it was on the same level as the buttons), thus I needed to add a little bit of extra padding:
Last but not least, I wanted to give certain UI elements a little bit of extra pizzaz. The headers in my table (named
So much for styling my app :-) It's probably not the most beautiful page in the world, but it was relatively painless and I can see myself using the password store in everyday life without getting annoyed by flashing colors and useless special effects.
No matter what one thinks of the quality of my UI design, it does show one thing though: using GWT makes building an ajaxy frontend as easy as building a scalable backend using App Engine. They are a great combination for building Apps, and I am looking forward to seeing what great applications others will release over the next couple of months!
Since this is probably going to be the last post in the Schlüsselmeister series, I'd like to start with a quick overview of what has been covered so far. The original goal had been to kick the tires of the newly releases Java version of Google's App Engine. I wanted to taste the Kool Aid and build an application using the newly released tools (GWT + Integrated Eclipse tool). Overall it has been a great experience and I surely learned a lot. Things that were covered in previous posts included:
- My first attempts at GWT programming and how it almost felt like being back in desktop software development.
- How coding against interfaces made almost my entire Javascript client testable through regular JUnit tests.
- How tools like EasyMock helped me test my classes in isolation, even if I had not coded all the components they depended on yet.
- How serialization that worked even across language barriers (from Java to Javascript and vice versa) made client-server communication almost trivial.
- How it is possible for apps with only small chunks of data (like Schlüsselmeister) to build a server that does not need a lot of complex JDO/JPA setup.
- How I used protocol buffers on the server and to build a backend that is easily unit-testable.
- How I was able to use a native Javascript crypto library in GWT.
- How I replaced my html landing page with JSP to turn the application off if no user was logged in.
It was a lot of fun -- until today, where I faced my ultimate enemy: Cascading Style Sheets! It is sad, but I think I could not come up with a good looking page on my own if my life depended on it. I needed something to get started with. I needed a template!
A few google searches later, I found this link on mycelly.com. I'm certainly not an artist or expert in usability, but I know what I like when I see it. The site seemed simple (not too many flashy colors), clutter-free, and the designer explicitely allowed people to use the stuff for one's own page (I don't want to rip anyone off after all!). It even had a little extra area at the top that I could use to add a tipjoy button ;-).
With the foundation for my site found, I adapted the JSP to fit the mold:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<%@page import="com.google.appengine.api.users.UserServiceFactory"%>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<link type="text/css" rel="stylesheet" href="Schluesselmeister.css">
<title>App Engine Fan's "Schlüsselmeister"</title>
<script type="text/javascript" language="javascript"
src="schluesselmeister/schluesselmeister.nocache.js"></script>
</head>
<body>
<div id="content">
<div id="navi">
<!-- TIPJOY BUTTON GOES HERE -->
</div>
<h1>App Engine Fan's <i>Schlüsselmeister</i></h1>
<h2>Password management software for the internets</h2>
<div id="about">
<!-- THIS IS WHERE I DESCRIBE HOW AWESOME MY SOFTWARE IS ;-) -->
<!-- IF A USER IS LOGGED ON, SHOW THE PROGRAM -->
<% if (UserServiceFactory.getUserService().getCurrentUser() != null) { %>
</div>
<div id="program" />
<!-- OTHERWISE, FILL THE VOID WITH A LITTLE BIT OF ADS -->
<% } else { %>
<p><b>To use this program, you have to <a
href="<%= UserServiceFactory.getUserService().createLoginURL("/") %>">log
in with your Google account.</a></b></p>
</div>
<div id="ad">
<i>(The following add will not be displayed once you are logged in.)</i><br/>
<!-- INSERT ADSENSE HERE -->
</div>
<% } %>
<!-- THAT's IT -->
</div>
</body>
</html>
I added the template to my stylesheet and fire up the application in Eclipse. So far, so good, but the program itself was still pretty bland! My solution to the problem is probably not the greates in the world, and if you have better ideas, please post them to the blog. What I did in this situation was: I opened up firebug.
Firebug is an awesome browser plugin that serves as a Javascript debugger, dom explorer and much more. One of the many cool things it does is its inspect functionality: it lets you select any section of a web page and shows its origin in the dom tree. I had just used GWT for the first time, and I had no idea (and no motivation to learn) how the web toolkit actually renders the UI Widgets into physical html. With Firebug, I could simply click and see what kind of element I had (was it a DIV? an INPUT? a SPAN?). Even better, the tool showed me exactly what the current style was, where the element "got" its style from (which css rules apply in which order? which are overwritten?). Last but not least, it also allowed me to change the style locally in the browser, which made it easy to try out how certain ideas look.
Back to styling my app. The first thing I did not like were how my dialogs looked. They did not have any borders, and their background was transparent (whatever text was behind the dialog would shine through). That had to be fixed! Fortunately, I had given both dialogs a style name in my GWT code, so I could easily address them in the dom tree:
.scramblerDialog, .editDialog {
border-style: ridge;
padding:1em;
background:white;
}
Next stop was the split panel. I wanted the splitter itself and the left hand side (the list of categories) to be in the same color, so that it would look like one coherent unit. The color should be a gray tone, since it seemed to fit well into the overall theme of the template. Unlike the previous example, I had not really set any style names in the code, so I had to do a little digging using Firebug:
.hsplitter, .splitPanel > div > div:first-child {
background:lightgrey;
}
I also realized that the list of categories was placed just a little bit too high (it was on the same level as the buttons), thus I needed to add a little bit of extra padding:
.gwt-Tree {
padding-top:1.5em;
}
Last but not least, I wanted to give certain UI elements a little bit of extra pizzaz. The headers in my table (named
header in my GWT code) were to get a little background color and padding. Similar styles should apply to the action elements (links a user would click on to edit a password entry or make a hidden password text readable). Here is the resulting set of rules:.header {
background: lightgrey;
padding-left:2px;
padding-right: 2px;
padding-bottom: 1px;
}
.action {
padding-left:0.2em;
padding-right:0.2em;
background:lightgrey;
color:blue;
}
So much for styling my app :-) It's probably not the most beautiful page in the world, but it was relatively painless and I can see myself using the password store in everyday life without getting annoyed by flashing colors and useless special effects.
No matter what one thinks of the quality of my UI design, it does show one thing though: using GWT makes building an ajaxy frontend as easy as building a scalable backend using App Engine. They are a great combination for building Apps, and I am looking forward to seeing what great applications others will release over the next couple of months!
Subscribe to:
Posts (Atom)
