Yes, we can. Not always correctly, of course. One way to predict things, is to try to find the probability distribution for a phenomenon. Usually a distribution is charted as a histogram, so that is what I did. I have a lot of ground to cover today, so I will be skipping some details here and there.

Services

I went a bit overboard with all kinds of Grails services. These are the services I made

• HistoricalQueryService – retrieves historical data from the database.
• FrequenceService calculates frequencies.
• SubtractService – subtracts vectors and calculates deltas (current – previous day).
• LnService – calculates ln var and ln var – ln var previous day.

Historical query service

The job of the historical query service is to get from the database historical data based on a symbol or symbols and a field name. For instance, DIA and Close. When two symbols are specified the service matches the data of the two instruments based on the added timestamp and subtracts the data. Here is the code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

    ...
    def queryVals(symbol,fieldName) {
       def instrument = Instrument.findBySymbol(symbol)
       def field = Field.findByName(fieldName)
       def fieldVals = FieldValue.findAllByInstrumentAndField(instrument, field)
       def data = []
 
      fieldVals.each {  data << Double.valueOf( it.val )}
 
      return data
    }
 
    def queryPairVals(symbol, symbol2, fieldName) {
    def instrument = Instrument.findBySymbol(symbol)
    def instrument2 = Instrument.findBySymbol(symbol2)
    def field = Field.findByName(fieldName)
    def fieldVals = FieldValue.findAllByInstrumentAndField(instrument, field)
    def fieldVals2 = FieldValue.findAllByInstrumentAndField(instrument2, field)
 
    def data = []
 
   for(int i = 0; i < fieldVals.size() && i < fieldVals2.size(); i++) {
      def sameDateVal = fieldVals2.each { 
   if(fieldVals[i].added == it.added) {
      return it.val
            }
         }
 
         if(sameDateVal == null) continue   
 
         data << Double.valueOf(fieldVals[i].val) - Double.valueOf(sameDateVal[0].val)
      }
 
      return data
    }
    ...

and here is the test code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

...
    void testQueryVals() {
      def type = new InstrumentType(type:"type", description:"desc")
      assertNotNull "Type should not be null", type.save()
 
      def ds = new Datasource(name:"DASOURCE", description: "desc")
      assertNotNull "DS should not be null", ds.save()
 
      def ins = new Instrument(name: "TEST", symbol: "TEST", 
         source: ds, instrumentType : type)
 
      assertNotNull "Instrument should not be null",ins.save()
 
      def f = new Field(name:"TOAST", description: "desc")
      assertNotNull "f should not be null", f.save()
 
      def v = new FieldValue(instrument:ins, field:f,
         added: new DateTime(), val: "1.0")
      assertNotNull "V should not be null", v.save()
 
      def vals = historicalQueryService.queryVals("TEST", "TOAST")
      assertNotNull "Vals should not be null",vals
      assertEquals "There should be 1 value", vals.size() , 1
    }
 
   void testQueryPairVals() {
      def type = new InstrumentType(type:"type", description:"desc")
      assertNotNull "Type should not be null", type.save()
 
      def ds = new Datasource(name:"DASOURCE", description: "desc")
      assertNotNull "DS should not be null", ds.save()
 
      def ins = new Instrument(name: "TEST", symbol: "TEST", 
         source: ds, instrumentType : type)
      def ins2 = new Instrument(name: "TEST2", symbol: "TEST2", 
         source: ds, instrumentType : type)
 
      assertNotNull "Instrument should not be null",ins.save()
      assertNotNull "Instrument should not be null",ins2.save()
 
      def f = new Field(name:"TOAST", description: "desc")
      assertNotNull "f should not be null", f.save()
 
      def now = new DateTime()   
      def v = new FieldValue(instrument:ins, field:f,
         added: now, val: "1.0")
      def v2 = new FieldValue(instrument:ins2, field:f,
         added: now, val: "3.0")
      assertNotNull "V should not be null", v.save()
      assertNotNull "V2 should not be null", v2.save()
 
      def vals = historicalQueryService.queryPairVals("TEST", "TEST2", "TOAST")
      assertNotNull "Vals should not be null",vals
      assertEquals "There should be 1 value", vals.size() , 1
   }
...

Frequence service

The frequence service defines a number of bins and counts the number of occurrences in each bin. I defined a Bin class as follows

1
2
3
4
5
6
7
8
9
10
11
12
13

...
   double start
   double end
   double mid
   String label
 
   Bin(s, e) {
      start = s
      end = e
      mid =  (start + end) / 2 
      label = sprintf('%.3g', mid)
   }
...

The service code is shown below. First I calculate the size of bin, then I make a list of Bin objects. After that I check in which bin each input value falls and increase the count for the lucky bin. There is also a convenience method that reduces the x labels by a factor of million – this is handy for volumes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

...
    def frequence(input) {
      def step = calcStep(input)
      def bins = makeBins(input.min(), step) 
      def frequency = [:]
 
      for( int i in 0..<NBINS) {
         frequency[ bins[i].label ] = 0
      }
 
      def size = input.size()
 
      input.each { 
         def index = calcIndex(it, bins)
         frequency[ bins[index].label]++
      }   
 
      return frequency
    }
 
   def makeBins(min, step) {
      def bins = []
 
      for( int i in 0..<NBINS) {
         bins << new Bin(min + i * step, min + (i + 1) * step)
      }
 
      return bins
    }
 
   def reduceKeys(frequency) {
      def reduced = [:]
 
      frequency.each { 
         key,value -> 
            reduced[Math.round(Double.valueOf(key)/(1000 * 1000))] = value
      }
 
      return reduced
   }
 
   def calcIndex(it, bins) {
      for(int i in 0..<bins.size()) {
         if(it >= bins[i].start && it < bins[i].end) {
            return i
         }
      }
 
      return bins.size() - 1 
   }
 
   def calcStep(input) {
      return (input.max() - input.min())/ NBINS
   }
...

Here is some of the test code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

...
   void testFrequence() {
      assertNotNull frequenceService
      def frequencies = frequenceService.frequence([1.0, 1.5, 32.0])
      assertNotNull frequencies
      assertEquals "Number of bins incorrect", 30, frequencies.size()   
 
      def store = 0
 
      frequencies.each { key,value -> store += value }
 
      assertEquals 3, store
    }
 
   void testStep() {
      assertEquals 1.0, frequenceService.calcStep([0,30.0])
   }
...

Subtract service

The subtract service subtracts vectors and calculates deltas (current – previous day).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

...
    def minPrev(array) {
      def subtracted = []
 
      for(int i in 1..<array.size() ) {
         subtracted << array[i] - array[i - 1]
      }
 
      return subtracted
    }
 
   def min(left, right) {
      def subtracted = []
 
      for(int i in 0..<array.size() ) {
         subtracted << array[i] - array[i - 1]
      }
 
      return subtracted
   }
...
   def testMinPrevious() {
      def subtracted = subtractService.minPrev([1, 2, 3])
      assertNotNull subtracted
      assertEquals 2, subtracted.size()
      assertEquals 1, subtracted[0]
      assertEquals 1, subtracted[1]
   }
...

Ln service

The Ln service calculates the natural logarithm of a variable vector and the difference of the ln of a variable and the ln of the previous variable value. There is a small problem with logarithms. You cannot take the logarithm of 0 or negative numbers. So I cheat a little by substituting negative numbers by their absolute value and avoiding 0.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

...
    def ln(numbers) {
      def result = []
 
      numbers.each {
         if(it != 0) 
            result << Math.log(Math.abs(it))
       }
 
      return result
    }
 
    def deltaLn(numbers) {
      def result = []
 
      for(int i in 1..<numbers.size()) {
         if(numbers[i] != 0 && numbers[i - 1] != 0)
            result << Math.log(Math.abs(numbers[i])) - Math.log(Math.abs(numbers[i-1]))
      }
 
      return result
    }
...

Some more tests

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

...
    void testLn() {
      def outcome = lnService.ln([-1,0, 1])
      assertNotNull outcome
      assertEquals  2, outcome.size()
      assertEquals  0, outcome[0]
    }
 
   void testDeltaLn() {
      def outcome = lnService.deltaLn([-1,0,1,1])
      assertNotNull outcome
      assertEquals  1, outcome.size()
      assertEquals  0, outcome[0]
   }
...

Controller

Services on their own don’t do much, so we need a controller to use the services and communicate with the views.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

...
   def calcFrequence(vals, fieldVar) {
      def frequence = frequenceService.frequence(vals)
 
      if(params.fieldVar == 'Volume') {
         frequence = frequenceService.reduceKeys(frequence)
      }
 
      return frequence
   }
 
   def calcDeltaFrequence(vals) {
      def subtracted = subtractService.minPrev(vals)
      return frequenceService.frequence(subtracted)
   }
 
   def calcLnFrequency(vals) {
      def lned = lnService.ln(vals)
      return frequenceService.frequence(lned)
   }
 
   def calcDeltaLnFrequency(vals) {
      def deltaLned = lnService.deltaLn(vals)
      return frequenceService.frequence(deltaLned)
   }
 
   def makeResponse(vals) {
      def frequence = calcFrequence(vals, params.fieldVar)
 
      def deltasFrequency = calcDeltaFrequence(vals)
 
      def lnFrequency = calcLnFrequency(vals)
 
      def deltaLnFrequency = calcDeltaLnFrequency(vals)
 
      return [data : frequence, 
         deltaData : deltasFrequency,
         lnData : lnFrequency,
         deltaLnData : deltaLnFrequency]
   }
 
    def index = { 
      if(params.fieldVar != null) {
         def vals = historicalQueryService.queryVals( params.select1, params.fieldVar);
         return makeResponse(vals)
      }
   }
 
    def pairs = { 
      if(params.fieldVar != null) {
         def vals = historicalQueryService.queryPairVals( 
            params.select1, params.select2, params.fieldVar);
         return makeResponse(vals)
      }
   }
...

Views

There are 2 views – the main view for a single symbol and a view for a pair of symbols.

Main view

The view consists of a form with 4 histograms beneath it. The form lets you choose a symbol and a field. The histograms are made with the Google Charts Grails plugin.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

<html>
   <head>
      <title>Historical data histogram</title>
   </head>
   <body>
      <table>
         <tr>
            <g:form>
            <td>
                  <g:select name="select1" from="${['DIA', 'SPY', 'GLD']}" value="${params.select1}"/>
            </td> 
            <td>
               <g:select name="fieldVar" from="${['Open', 'High', 'Low', 'Close', 'Volume']}" value="${params.fieldVar}"/>
            </td>
            <td>
               <g:actionSubmit name="doChart" action="index" value="Chart"/>
            </td>
            </g:form></td>
         </tr>
      </table>
 
      <g:if test="${data != null}">
           <g:barChart type="bvs"
                title="Histogram ${params.select1} (${params.fieldVar})"
            size="${[900,200]}" 
            colors="${['00FF00']}" 
            fill="${'bg,s,efefef' }"
            axes="x,y"
            axesLabels="${
               [0:data.keySet(),
               1:[0,data.values().max()/2, data.values().max()]
               ]
                }"
            dataType="simple"
            data="${data.values().asList()}"
            />
      </g:if>
 
      <br/> <br/>
 
      <g:if test="${deltaData != null}">
           <g:barChart type="bvs"
                title="Histogram ${params.select1} (${params.fieldVar} - previous day ${params.fieldVar}) "
            size="${[900,200]}" 
            colors="${['00FF00']}" 
            fill="${'bg,s,efefef' }"
            axes="x,y"
            axesLabels="${
               [0:deltaData.keySet(),
               1:[0,deltaData.values().max()/2, deltaData.values().max()]
               ]
                }"
            dataType="simple"
            data="${deltaData.values().asList()}"
            />
      </g:if>
 
      <br/> <br/>
 
      <g:if test="${lnData != null}">
           <g:barChart type="bvs"
                title="Histogram ${params.select1} (Log ${params.fieldVar}) "
            size="${[900,200]}" 
            colors="${['00FF00']}" 
            fill="${'bg,s,efefef' }"
            axes="x,y"
            axesLabels="${
               [0:lnData.keySet(),
               1:[0,lnData.values().max()/2, lnData.values().max()]
               ]
                }"
            dataType="simple"
            data="${lnData.values().asList()}"
            />
      </g:if>
 
      <br/><br/>
 
      <g:if test="${deltaLnData != null}">
           <g:barChart type="bvs"
                title="Histogram ${params.select1} (Log ${params.fieldVar} - Log previous day ${params.fieldVar}) "
            size="${[900,200]}" 
            colors="${['00FF00']}" 
            fill="${'bg,s,efefef' }"
            axes="x,y"
            axesLabels="${
               [0:deltaLnData.keySet(),
               1:[0,deltaLnData.values().max()/2, deltaLnData.values().max()]
               ]
                }"
            dataType="simple"
            data="${deltaLnData.values().asList()}"
            />
      </g:if>
    </body>
</html>

The pairs view

The pairs view is almost a copy of the main view. The difference being that in this view you can specify an extra symbol.

Result

The result is displayed in the table below for GLD, SPY, DIA and their respective differences. The data comes from historical data of end of day OHLC prices and volume.

	Open	High	Low	Close	Volume
DIA
SPY
GLD
DIA – SPY
DIA – GLD
SPY – GLD

Conclusions

Turns out that the distributions are pretty non random, almost Gaussian. So we may be able to predict stock prices, a little a bit more accurately than you would have expected.

I should have started with Grails services from the beginning. Services are just another layer in the architecture. Services can easily be injected, wherever you need them. I also ran into some trouble, because of upgrading to a new Grails version.

Creating services

So I did a Mac Ports upgrade, which included upgrade of Grails to 1.3.4. When I tried to create a new service with

1

grails create-service simulation

Grails complained about a version mismatch, so I had to do

1

grails update

to fix that. The service

1
2
3
4
5
6
7
8
9
10
11

class SimulationService {
 
    static transactional = true
 
    def create(name, description) {
	def simulation = new Simulation(name, description)
	simulation.save()
 
	return Simulation.get(simulation.id)
    }
}

can now be injected into the controller for example.

1
2
3
4
5
6
7
8

class SimulationController {
   def scaffold = true
	def simulationService
 
	def persist(name, description)  {
		def simulation = simulationService.save(name, description)
	}
}

Testing services

Grails created a unit test stub along with the service, but I also wanted an integration test.

1

grails create-integration-test SimulationServiceIntegration

Gant then suddenly reported a MissingPropertyException. Very mysterious. It turns out this is a known bug/issue GRAILS-6606. The fix was easy – adding the line

1

def type = "Tests"

to CreateIntegrationTest.groovy

1
2
3
4
5
6
7
8
9
10
11

...
target ('default': "Creates a new Grails integration test which loads the whole Grails environment when run") {
    depends(checkVersion, parseArguments)
 
    def type = "Tests"
    promptForName(type: "Integration test")
 
    def name = argsMap["params"][0]
    name = purgeRedundantArtifactSuffix(name, type)
    createIntegrationTest(name: name, suffix: "")
}

The bad news is, that there are bugs/issues in Grails 1.3.4. The good news is that, some of these are easy to fix. Here is an example test

1
2
3
4
5
6

   ...
   def simulationService
 
    void testCreate() {
	assertNotNull simulationService
    }

Dependency Injection rules.

You can run all the tests with

1

grails test-app

Side topic: Levy distribution

It is a well known fact that stock prices etc. do not conform to the normal distribution. Apparently some people have the idea that the Levy distribution is more appropriate. One of these people is Mandelbrot of the fractals. Mandelbrot recognizes four schools of thought

1. The most popular, technical analysts school exploiting short term changes.
2. Random walk, statistically independent prices, but price changes are Gauss distributed.
3. Another random walk believers school, but price changes have infinite variance.
4. Random walk is a first order approximation, but there is also a second order!

This blog explains why the Levy distribution makes trend following profitable. Let the profits run and cut your losses. If you do it correctly, the fat tail of the distribution will make you a net profit.

Random links of interest

• Plugging memory leaks with weak references
• Command-line flags for the JVM
• The Financial Markets Seismometer
• C9 lectures – functional programming intro
• Cloudera Hadoop Training
• Half arsed agile manifesto

Change log

The new features are

• Persistent undo and undo for reload
• Blowfish encryption, encryption of the swap file
• Conceal text
• Lua interface
• Python 3 interface

1

:help version-7.3

offers more detailed information. So I downloaded the latest version of MacVim and installed vim with MacPorts

1

sudo port install vim

I checked the enabled features with :version and

1

vim --version

I noticed that the Python 3 and Lua interface are disabled.

Undo persistence

Undo persistence is a new vim feature. You can find information about undo persistence with

1

:help undo-persistence

In my .vimrc I put these lines

1
2

set undofile
set undodir=/tmp/undos

The first line turns on undo persistence, which is off by default. The second line specifies a directory where to store the undo files, otherwise vim would use the same directory where the edited file is.

Bag of tips and tricks (pre 7.3)

My bag of vim gems and tidbits is full of stuff I would like to share no matter what. I admit that the usefulness varies and some of this stuff you may never ever need.

XML folding

I think that XML folding is a really neat feature. The xmledit vim plugin lets you do this the easy way. Download the plugin, put it in the .vim/ftplugin directory and add these lines to .vimrc

1
2

let g:xml_syntax_folding=1
au FileType xml setlocal foldmethod=syntax

If you open a XML file, it will be folded already. You can close and open folds with zo and zc.

Save a file with sudo

1

:w !sudo tee %

The ! calls an external command – in this case tee. % is the vim register containing the name of the file.

Encoding files

Vim allows you to encode data with a password

1

vim -x secretfile

Now vim asks you to enter an encryption key also known as a password and then you need to repeat it. Next time you try to open this file you will be prompted for your password. Of course, if you try viewing the file with other editors or cat for example, you will only see gibberish.

MySQL shell vi mode

I tested this under Ubuntu only, but in theory it should work on other operating systems as well. Apparently if MySQL is compiled with readline support, you can set vi mode for the MySQL CLI. Add this line to ~/.inputrc

1
2

set keymap vi 
set editing-mode vi

Vim questions on Stack Overflow

I assembled a list of vim questions on Stack Overflow. The answers contain lots of useful tips and ideas. I highly recommend going through them. The signal to noise ratio is really good.

• Favorite vim plugins
• Making vim ubiquitous apparently evince let you use hjkl as well
• What is in your vimrc
• The dark corners of vim
• Most productive shortcut with vim

Random links of interest

As a free service here are some interesting links

• Cassandra Summit 2010
• Terminal Tips and Tricks For Mac OS X
• vi tips and tricks

Test a little, code a little, go home, eat a little, sleep a little. This is the lifestyle recommended by the League of Agile Methodology Experts. That and wearing t-shirts with agile slogans printed on them and matching pants. I would like to do that today, the testing part I mean.

Integration tests

Grails make it easy to create integration tests. Let’s do that real quick

1

grails create-integration-test SimulationIntegration

I modified the test to create and delete a Simulation object on the fly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

import grails.test.*
 
class SimulationIntegrationTests extends GrailsUnitTestCase {
    protected void setUp() {
        super.setUp()
    }
 
    protected void tearDown() {
        super.tearDown()
    }
 
    void testSaveAndDelete() {
		def simulation = new Simulation(name : "sim1", description : " first test")
		assertNotNull simulation.save()
 
		def foundSimulation = Simulation.get(simulation.id)
		foundSimulation.delete()
 
		assertFalse Simulation.exists(foundSimulation.id)
    }
}

You can run the test with

1

grails test-app -integration

Another integraion test

The problem with having many domain classes is that you need many tests. LAME practicioners use the word little, because otherwise testing gets boring. If you have the code already as I do, then it becomes test a little, then test a little more, which is not recommended at all. Sigh, here is another test

1

grails create-integration-test SessionIntegration

The test adds a few sessions to the simulation. Note the use of the dynamic addTo method.

1
2
3
4
5
6
7
8
9
10
11
12

    void testFirstSimulation() {
		def simulation = new Simulation(name : "sim1", description : " first test").save()
		def session1 = new Session(description : "First session")
		simulation.addToSessions( session1 )
		def session2 = new Session(description : "Second session")
		simulation.addToSessions( session2 )
		assertEquals 2, Simulation.get(simulation.id).sessions.size()
 
		def foundSimulation = Simulation.get(simulation.id)	
		def descriptions = foundSimulation.sessions.collect { it.description }
		assertEquals( ['First session', 'Second session'], descriptions.sort())
    }

Intermission

Let’s get something to drink. I have been researching trading concepts and approaches lately. So this is my list of popular ideas:

• Trend following – determine when a market is trending and follow it until the trend vanishes.
• Hedging – lower investment risk with short selling and/or derivatives.
• Fundamental analysis
• Value investing – invest based on intrinsic value determination.
• Band trading – trade within a band.
• Spreads – buying and selling different legs of an option or future.
• Arbitrage – taking advantage of price differences between different markets.

OK, back to the main program.

Yet another integration test

Yes, more tests, but being LAME is not easy.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

    void testSessionRunCreation() {
		def simulation = new Simulation(name : "sim1", description : " first test").save()
		def session = new Session(description : "First session")
 
		def run1 = new SessionRun()
		def param1 = new StrategyParameter( name : "param1", 
			description : "first parameter", val : "1.0")
		run1.addToParameters( param1 )
		session.addToRuns( run1 )
		assertEquals 1, session.runs.size()
 
		def run2 = new SessionRun()
		def param2 = new StrategyParameter( name : "param2", 
			description : "second parameter", val : "2.0")
		run2.addToParameters( param2 )
		session.addToRuns( run2 )
		assertEquals 2, session.runs.size()
 
		simulation.addToSessions( session )
 
		assertEquals ([ 'param1' , 'param2' ],	
			simulation.sessions.runs.parameters*.name.flatten().sort())
 
		def foundSimulation = Simulation.get(simulation.id)
		assertEquals ([ 'param1' , 'param2' ],	
			foundSimulation.sessions.runs.parameters*.name.flatten().sort())
    }

Random links of interest

I would like to apologize for the boring stuff above. Please accept these links of interest as compensation.

• GPU Gems 2 book a free online book on GPU programming.
• GPGPU wiki
• Stanford University courses on YouTube.
• GPars Groovy Parallel Systems.
• Groovy++ statically typed extension of Groovy.



]]> http://ivanidris.net/wordpress/index.php/2010/08/15/grails-finance-0-6-testing/feed 0 http://ivanidris.net/wordpress/index.php/2010/08/08/grails-finance-0-5 http://ivanidris.net/wordpress/index.php/2010/08/08/grails-finance-0-5#comments Sun, 08 Aug 2010 17:45:02 +0000 admin http://ivanidris.net/wordpress/index.php/2010/08/08/grails-finance-0-5 Birth of a simulation engine

Mmm, okay, so it’s back to the drawing board for Design Phase Deux. I want to run trading simulation sessions using different strategies. Each session will consist of different runs, where certain parameter values will be varied. Each run will have a P & L, that will be stored for later analysis. The simulation engine will generate orders, which will have different status and type.

Class diagram

I did some CRC analysis in my head and sketches on paper. So I made a preliminary draft design, which is obviously subject to change. Unfortunately, I don’t know how to draw UML diagrams in Vim. After trying out a few tools and getting increasingly annoyed I settled for Netbeans, because its diagrams seemed prettiest. Netbeans 6.5 to be exact, not the latest version, but the version I already had installed. Here is the class diagram, some of the datatypes are incorrect and while writing about the design, I realized, that I forgot a few classes



Code generation

Netbeans has the ability to generate code for you from UML, but it requires you to make a target project. Since I am not a Netbeans fan, I decided to generate the code myself. For that purpose I generated a HTML report from Netbeans. The script below parses the HTML files, calls the Grails commands to generate controllers and domain classes and then appends variable declarations to the generated domain classes. This means that manual editing is still required.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

REPORTDIR=~/NetBeansProjects/SimulationEngine/report/SimulationEngine   for f in `ls $REPORTDIR/*.html` do str=`grep "public class" $f 2>/dev/null| grep -v Unnamed| grep "<B>"`   if [ -n "$str" ] then class=`echo $str|sed -e 's/.*<B>//g;s/<\/B>.*//g'` grails create-controller $class grails create-domain-class $class varname=`grep private $f| sed -e 's/.*html">//g;s/<\/A>//g;s/<B>//g;s/<\/B>.*$//g'` echo " $varname" >> grails-app/domain/finance/${class}.groovy fi done

Manual editing

The quick and dirty code generation script created code that doesn’t compile. I had to move the variable declarations to the correct place, fill in the constraints, belongsTo and hasMany values. Also it is good practice to have a toString implementation. The current stats are as follows

1 2 3 4 5 6 7 8 9 10 11

grails stats   +----------------------+-------+-------+ | Name | Files | LOC | +----------------------+-------+-------+ | Controllers | 15 | 174 | | Domain Classes | 14 | 168 | | Unit Tests | 29 | 348 | +----------------------+-------+-------+ | Totals | 58 | 690 | +----------------------+-------+-------+

690 LOC, not bad for a SHH ( Secret Home Hackathon ).

Random links of interest

And … now something really, absolutely and completely different.

• CodingBat – webbased you can create an account here and track your progress.
• SingPath – also webbased, track progress by signing in with your Google account.
• Python koans – you can download the exercises with Mercurial. The exercises are failing tests, that you need to make pass.
• Scala days 2010 videos
• Vrapper – Eclipse vim plugin. Haven’t tried it out yet. TODO



]]> http://ivanidris.net/wordpress/index.php/2010/08/08/grails-finance-0-5/feed 0 http://ivanidris.net/wordpress/index.php/2010/08/01/grails-finance-0-4 http://ivanidris.net/wordpress/index.php/2010/08/01/grails-finance-0-4#comments Sun, 01 Aug 2010 17:12:51 +0000 admin http://ivanidris.net/wordpress/index.php/2010/08/01/grails-finance-0-4 Approved by the League of Agile Methodology Experts

Like, let me go ahead and confirm the results of Grails Finance 0.3. In statistics there is the notion of correlation. In the case of a non normal distribution, it seems most appropriate to calculate the Spearmans correlation coefficient. The coefficient varies between -1 and 1. 0 corresponds to independence, the closer the coefficient is to -1 or 1 the more correlation there is. I will use the Apache Commons Math API to determine the correlation.

Dependencies

The whole world uses Maven to manage dependencies with the principle of convention over configuration. Grails also uses this principle, but with a different convention and less XML. If you really want to, you can integrate Grails and Maven using a Maven archetype or special plugins (see Grails configuration). Instead I chose to tweak BuildConfig.groovy – the repositories and dependencies blocks.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

repositories { grailsPlugins() grailsHome() grailsCentral()   // uncomment the below to enable remote dependency resolution // from public Maven repositories mavenLocal() mavenCentral() mavenRepo "http://snapshots.repository.codehaus.org" mavenRepo "http://repository.codehaus.org" mavenRepo "http://download.java.net/maven/2/" mavenRepo "http://repository.jboss.com/maven2/" } dependencies { // specify dependencies here under either 'build', 'compile', 'runtime', 'test' or 'provided' scopes eg. runtime 'org.apache.commons:commons-math:2.1' // runtime 'mysql:mysql-connector-java:5.1.5' }

You can test the dependencies with

1	grails dependency-report

Controller

I added some logic in the controller to calculate correlations of a specified instrument pair for OHLC and volume.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

import org.apache.commons.math.stat.correlation.SpearmansCorrelation   ...   def correlation = { if(params.select1 != null) { return [instruments : getGoogleInstruments(), open: correlate(params.select1, params.select2, 'Open'), high: correlate(params.select1, params.select2, 'High'), low: correlate(params.select1, params.select2, 'Low'), close: correlate(params.select1, params.select2, 'Close'), volume: correlate(params.select1, params.select2, 'Volume') ] } else { return [instruments : getGoogleInstruments()] } }   def correlate(symbol1, symbol2, fieldName) { def instrument = Instrument.findBySymbol(symbol1) def instrument2 = Instrument.findBySymbol(symbol2) def field = Field.findByName(fieldName) def firstData = getDataForInstrument(instrument, field) def secondData = getDataForInstrument(instrument2, field)   def data1 = [] def data2 = []   for(def added : firstData?.keySet()){ if( secondData?.keySet()?.contains(added)) { data1 << Double.valueOf(firstData?.get(added)) data2 << Double.valueOf(secondData?.get(added)) } }   def corr = new SpearmansCorrelation().correlation((double[])data1.toArray(), (double[])data2.toArray()) }

View

The view displays a correlation table for an instrument pair.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

<html> <head> <title>Correlation Historical data</title> </head> <body> <table> <tr> <g:form> <td> <g:select name="select1" from="${instruments.symbol}" value="${params.select1}"/> </td> <td> <g:select name="select2" from="${instruments.symbol}" value="${params.select2}"/> <td> <td> <g:actionSubmit name="correlation" action="correlation" value="Correlate"/> </td> </g:form></td> </tr> </table>   <br> <br>   <g:if test="${open != null}"> <table border="1"> <tr> <td>Open</td> <td>${open}</td> </tr> <tr> <td>High</td> <td>${high}</td> </tr> <tr> <td>Low</td> <td>${low}</td> <tr> <td>Close</td> <td>${close}</td> </tr> <tr> <td>Volume</td> <td>${volume}</td> </tr> </table> </g:if> </body> </html>

Result

The results confirm the expectations. DIA and SPY prices have strong correlation. The correlation with GLD is weaker. Also volumes have consistently lower correlations, but still higher than 0.





Conclusion

The results lead me to believe, that I have found some dependencies, whether they are real remains to be seen. One way to check that, is to do some backtesting. For that I will need to build a simulation engine, which will require a complete application redesign and refactoring. You might be wondering what the plans exactly are. As LAME practicioners tend to say, there are no plans, but certainly there are lots of hopes and dreams.

Random links of interest

And now for something completely different – a list of totally unrelated links of awesome importance and content. Probably. Anyway, if you have a link of relevance, that you want to add, feel free to share it in the comments.

• Google IO 2010 Google’s yearly conference for developers.
• Google Code University contains complete courses and video lectures.
• Khan Academy haven’t checked it out, but it gets good press. TODO.
• Readability bookmarlet, well name says it all.
• SpringSource free online training not a lot there, but good quality.



]]> http://ivanidris.net/wordpress/index.php/2010/08/01/grails-finance-0-4/feed 0 http://ivanidris.net/wordpress/index.php/2010/07/25/grails-finance-0-3 http://ivanidris.net/wordpress/index.php/2010/07/25/grails-finance-0-3#comments Sun, 25 Jul 2010 19:59:57 +0000 admin http://ivanidris.net/wordpress/index.php/2010/07/25/grails-finance-0-3 So now that I have historical data in my database, it’s time to have a good look at it. One popular way is to represent the data in a chart. Creating graphs on your own from scratch is a lot of work. Luckily, there are a lot of freely available charting API’s out there. For some of those API’s Grails plugins exist.

Google Charts

Google Charts generates an image from an URL with parameters. The Google Chart Grails plugin has a set of tags with, which you can specify what kind of chart you want. Install it

1	grails install-plugin google-chart

The testing procedure on the plugin information page did not work for me and it seems that the plugin does not support all the Google chart types. Other than that, everything seems to work fine.

Controller

I added two methods in the controller. The first one, gets historical end of day data sorted by date, for an instrument and a field. For instance, DIA and the high price field. The second method calls the first one for two different securities and the same field in order to compare the data in a chart. This method also produces a list of x axis labels.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

def historicalChart = { if(params.fieldVar == null) { return [instruments : getGoogleInstruments()] } else { def instrument = Instrument.findBySymbol(params.select1) def instrument2 = Instrument.findBySymbol(params.select2) def field = Field.findByName(params.fieldVar) def firstData = getDataForInstrument(instrument, field) def secondData = getDataForInstrument(instrument2, field)   def data1 = [:] def data2 = [:] def xLabels = []   for(def added : firstData?.keySet()){ if( secondData?.keySet()?.contains(added)) { def shortKey = added.toString("MMM yy")   if(!xLabels.contains(shortKey)) { xLabels << shortKey }   data1[added] = Double.valueOf(firstData?.get(added)) data2[added] = Double.valueOf(secondData?.get(added)) } }   return [instruments : getGoogleInstruments(), firstData : data1, secondData : data2, xLabels : xLabels] } }   def getDataForInstrument(instrument, field) { def fieldVals = FieldValue.findAllByInstrumentAndField(instrument, field) def data = [:]   for(int i = 0; i < fieldVals.size(); i++) { data[fieldVals[i].added] = fieldVals[i].val }   return data.sort { it.key } }

The view

The view for the chart consists of one GSP page. I put some logic in it, that possibly could have been better placed in the controller. The page consists roughly of two parts. The top part is a form, which lets you select two instruments and a field, such as a price to plot. The bottom part displays a line chart with a special tag.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

<html> <head> <title>Chart Historical data</title> </head> <body> <table> <tr> <g:form> <td> <g:select name="select1" from="${instruments.symbol}" value="${params.select1}"/> </td> <td> <g:select name="select2" from="${instruments.symbol}" value="${params.select2}"/> <td> <td> <g:select name="fieldVar" from="${['Open', 'High', 'Low', 'Close', 'Volume']}" value="${params.fieldVar}"/> </td> <td> <g:actionSubmit name="doChart" action="historicalChart" value="Chart"/> </td> </g:form></td> </tr> </table>   <g:if test="${firstData != null}"> <% def firstMax = firstData?.values()?.max() def secondMax = secondData?.values()?.max() def yMax = Math.max(firstMax, secondMax)   def priceData = [firstData?.values()?.asList(), secondData?.values()?.asList()] %> <g:lineChart type="lc" title="End of day (${params.fieldVar}) comparison" size="${[600,200]}" colors="${['FF0000','0000FF']}" axes="x,y" lineStyles="${[[2,2,2],[2,8,4]]}" legend="${[ params.select1, params.select2 ]}" gridLines="${100.0/11},25" axesLabels="${[0:xLabels,1:[0,yMax/2,yMax]] }" data="${priceData}" dataType="simple" /> </g:if> </body> </html>

Result

The result of this little science project is a table of charts. I plotted DIA, a Dow Jones Industrial Average tracker, SPY, a Standard and Poors 500 tracker and GLD, a Gold Trust tracker against each other for OHLC and volume parameters. The underlying indices of the first two trackers have similarities, therefore I expect, that the trackers wil be also linked in some way. GLD was added for good measure, because as 9 out of 10 scientists will, tell you, gold is very shiny .

	DIA x SPY	DIA x GLD	SPY x GLD
Open
High
Low
Close
Volume

Conclusion

As expected there seems to be a relation between the DIA and SPY prices. Volumes differ by orders of magnitudes, which makes it hard to see any pattern in the charts. I can not say much about GLD. Next time I will perform simple statistical analysis to quantify any possible correlation. Oh, yeah, Grails is great and plugins too.



]]> http://ivanidris.net/wordpress/index.php/2010/07/25/grails-finance-0-3/feed 0 http://ivanidris.net/wordpress/index.php/2010/07/18/grails-finance-0-2 http://ivanidris.net/wordpress/index.php/2010/07/18/grails-finance-0-2#comments Sun, 18 Jul 2010 17:04:08 +0000 admin http://ivanidris.net/wordpress/index.php/2010/07/18/grails-finance-0-2 Last week I made a scaffolded Grails finance application. This week I will fill the database with historical price data from Google Finance.

Model Changes

I needed to change the model a bit. I changed the name of the DateTime field and added some instance variables with corresponding constraints. Here is the modified FieldValue domain class

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

package finance   import org.joda.time.* import org.joda.time.contrib.hibernate.*   class FieldValue { String val DateTime added Instrument instrument Field field   static constraints = { val(blank:false) added(blank:false) instrument(blank:false) field(blank:false) }   static belongsTo = [Instrument, Field]   String toString() { "${val} ${added}" } }

The view

I created a Groovy Server Page (GSP) that displays in a table all the Google Finance instruments in the database and buttons for data retrieval. Prototype is used to perform AJAX calls to the controller. I hope you recognize the famous Spinner UI design pattern.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

<html> <head> <title>Retrieve EOD data from Google Finance</title> <g:javascript library="prototype"/> </head> <body> <table> <g:each var="instrument" in="${instruments}"> <tr> <td>${instrument.symbol}</td> <td>${instrument.name}</td> <td><g:form action='retrieve'> <g:hiddenField name='symbol' value='${instrument.symbol}'/> <g:submitToRemote value="Get data" url="[controller: 'fieldValue', action: 'retrieve']" onLoading="showSpinner(true)" onComplete="showSpinner(false)"/> <img id="spinner" style="display: none" src="<g:createLinkTo dir='/images' file='spinner.gif'/>"/> </g:form></td> </tr> </g:each> </table> <g:javascript> function showSpinner(visible) { $('spinner').style.display = visible ? "inline" : "none" } </g:javascript>   </body> </html>

The page looks like this in Safari



The controller

The controller does all the hard work of querying for instruments and retrieving data in CSV format. The data consists of a Date, Open, HighLow, Close and Volume column. I installed the REST plugin for the HTTP communication part

1	grails install-plugin rest

The plugin injects amongst others the withAsyncHttp method.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

package finance   import static groovyx.net.http.ContentType.HTML import org.joda.time.* import org.joda.time.format.* import org.joda.time.contrib.hibernate.*   class FieldValueController { def scaffold = true   def eodgoogle = { def instruments = Instrument.findAll().findAll { instrument -> instrument.source.name == 'Google Finance' } [instruments : instruments] }   def retrieve = { withAsyncHttp(poolSize : 4, uri : "http://finance.google.com", contentType : HTML) { def result = get(path:'/finance/historical', query: [q:params.symbol, output:'csv']) { resp, html -> println ' got async response!' return html }   assert result instanceof java.util.concurrent.Future   while (! result.done) { Thread.sleep(2000) }   def html = result.get().toString()   for(line in html.split('\n')) { if(!line.contains('Date')) { def fields = line.split(',') def dateTime = DateTimeFormat.forPattern("dd-MMM-yy").parseDateTime(fields[0]) def instrument = Instrument.findBySymbol(params.symbol) saveValueByFieldName('Open', dateTime, instrument, fields[1]) saveValueByFieldName('High', dateTime, instrument, fields[2]) saveValueByFieldName('Low', dateTime, instrument, fields[3]) saveValueByFieldName('Close', dateTime, instrument, fields[4]) saveValueByFieldName('Volume', dateTime, instrument, fields[5]) } } } }   def saveValueByFieldName(fieldName, dateTime, instrument, val) { def mnemonic = Field.findByName(fieldName) def fVal = new FieldValue(added: dateTime, val: val, instrument: instrument, field:mnemonic).save(); } }

A screenshot of the historical values list is shown here below. Mmm, gold …



Notice that no effort is required to get pagination.

Conclusion

Grails is really perfect for secret home projects, especially for weekend ones. Next on the Grails finance roadmap is charting, some mathematical analysis, a REST webservice, lots of polishing and even more features, which will remain secret for now.



]]> http://ivanidris.net/wordpress/index.php/2010/07/18/grails-finance-0-2/feed 0 http://ivanidris.net/wordpress/index.php/2010/07/11/grails-finance-0-1 http://ivanidris.net/wordpress/index.php/2010/07/11/grails-finance-0-1#comments Sun, 11 Jul 2010 20:23:25 +0000 admin http://ivanidris.net/wordpress/index.php/2010/07/11/grails-finance-0-1 Grails is an automagical RAD web application framework in the style of Ruby on Rails. Grails uses Groovy as its main language. You could also substitute it with Java, but that would be less fun. As an example I made a small web application for financial data.

Installation

In Mac Ports this command installs Grails

1	sudo port install grails

to be exact it installs version 1.3.2 Just this week version 1.3.3 was released. You still need to put Grails in your path

1 2	export GRAILS_HOME=/opt/local/share/java/grails export PATH=$PATH:$GRAILS_HOME/bin

Creating the app

Grails lets you generate code and configuration files from the command line or if you are a wimp you can use an IDE instead. This command generates the application

1 2	grails create-app finance cd finance

You can already run the application,

1	grails run-app

but of course, there is not much to see yet.

Grails MVC

Grails reinvents MVC and provides scripts for it. Here is how we can create a controller

1 2 3 4 5 6 7 8 9 10 11 12

grails create-controller finance   Resolving dependencies... Dependencies resolved in 1439ms. Running script /opt/local/share/java/grails/scripts/CreateController.groovy Environment set to development [mkdir] Created dir: ... finance/grails-app/controllers/finance Created Controller for Finance [mkdir] Created dir: ... finance/grails-app/views/finance [groovyc] Compiling 1 source file to /Users/ivanidris/Projects/grails/finance/target/classes [mkdir] Created dir: ... finance/test/unit/finance Created Tests for Finance

Grails generates some tests for the lazy Agile developer and a controller

1 2 3 4 5

package finance   class FinanceController { def index = { } }

Now we can create some entity model classes or as Grails calls them domain classes

1	grails create-domain-class datasource

The datasource class will store information for the financial data sources. They will all be free, of course, since this is a home project.

1 2 3 4 5 6 7 8 9

package finance   class Datasource { String name String description   static constraints = { } }

Tweaking

In grails-app/conf/DataSource.groovy you can find configuration settings for different environments such as test, development and production. I changed the development settings so that the database will be persisted to a file for now. Every time Grails notices a change in the code the server is restarted. I changed the dbCreate parameter otherwise with the initial value of create-drop the database would have been dropped.

1 2 3 4 5 6

development { dataSource { dbCreate = "update" // one of 'create', 'create-drop','update' url = "jdbc:hsqldb:file:devDB;shutdown=true" } }

Scaffolding feature

Scaffolding gives you basic CRUD screens for your domain classes. I noticed a feature, when I setup scaffolding for Datasource in the main controller.

1 2 3 4 5 6

package finance   class FinanceController { def scaffold = Datasource def index = { } }

The following stack trace was produced

ERROR view.ScaffoldingViewResolver  - Error generating scaffolded view [/finance/index]: /opt/local/share/java/grails/src/grails/templates/scaffolding/index.gsp (No such file or directory)
java.io.FileNotFoundException: /opt/local/share/java/grails/src/grails/templates/scaffolding/index.gsp (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.(FileInputStream.java:106)
	at java.lang.Thread.run(Thread.java:637)

The error message although correct was not very helpful. The fix was to change the controller to this

1 2 3 4 5

package finance   class FinanceController { def scaffold = Datasource }

Constraints

You can define constraints for your domain classes. For instance, here name is required to be not null or blank. There are a few other possible constraints, such as defining a field to be an email address or URL.

1 2 3 4 5 6 7 8 9 10 11 12

package finance   class Datasource { String name String description Date added = new Date()   static constraints = { name(blank:false) description(maxSize:255) } }

More commands

At a certain point you will want to make a WAR

1 2 3

grails war   Done creating WAR ... finance/target/finance-0.1.war

Also you might be interested in how many files and lines of code were written

1 2 3 4 5 6 7 8 9 10 11

grails stats   +----------------------+-------+-------+ | Name | Files | LOC | +----------------------+-------+-------+ | Controllers | 1 | 4 | | Domain Classes | 1 | 10 | | Unit Tests | 2 | 24 | +----------------------+-------+-------+ | Totals | 4 | 38 | +----------------------+-------+-------+

Not bad for a few minutes work. Strange that the configuration files and views are not listed.

Instrument type

Now it’s time to get serious about the domain model. Well not production type serious, more secret home weekend project type serious . First let’s define InstrumentType. Every instrument would have a type, unless of course the type is unknown. The 1:1 relation description is set in belongsTo. The type is set to be unique. It is recommended to have a toString implementation, otherwise Grails will display the classname and record id by default in the UI.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

package finance   class InstrumentType { static belongsTo = Instrument   String type String description   static constraints = { type(unique:true, blank:false) }   String toString() { "${type}" } }

Instrument

The amazing thing about the financial industry, is that, there are no real identification standards. You would think, that for such a multi billion type business, some general scheme would exist. Each data vendor has their own symbology so for the same instrument we would have different records. One for each data source.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

package finance   class Instrument { String name String symbol InstrumentType instrumentType Date added = new Date() Datasource source     static constraints = { name(unique:true, blank:false) symbol(unique:true, blank:false) source(blank:false) }   static mapping = { instrumentType lazy:false }   static hasMany = [ contains:Instrument, values:FieldValue]   String toString() { "${name} ${symbol} ${source}" } }

We can have the instrument type be lazy fetched or in this case eagerly. hasMany here indicates that an instrument can contain other instruments and could have many values.

Field

A field could be an attribute that does not change a lot or a price. It might be useful to have a field type domain class as well, but I decided to keep things simple for now.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

package finance   class Field { String name String description   static constraints = { name(blank:false) }   static hasMany = [ values : FieldValue ]   String toString() { "${name}" } }

FieldValue

Value is a keyword I think, just like values is a SQL keyword. Value is not allowed by groovyc

Apparent variable 'Value' was found in a static scope but doesn't refer to a local variable, static field or class. Possible causes:
  [groovyc] You attempted to reference a variable in the binding or an instance variable from a static context.
  [groovyc] You misspelled a classname or statically imported field. Please check the spelling.
  [groovyc] You attempted to use a method 'Value' but left out brackets in a place not allowed by the grammar.
  [groovyc]  @ line 11, column 33.
  [groovyc]        static hasMany = [ values : Value ]
  [groovyc]                                    ^
  [groovyc]
  [groovyc] 1 error

For the field values we need a proper timestamp. java.util.Date is not good enough for that so we will need Jodatime. Fortunately Grails has a whole plugin ecosystem, including a Jodatime plugin that we can install.

1 2 3

grails install-plugin joda-time grails install-joda-time-gorm-mappings grails install-joda-time-templates

The value is stored as a String. This is not type safe, so it would have been better to have a value table for each data type.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

package finance   import org.joda.time.* import org.joda.time.contrib.hibernate.*   class FieldValue { String val DateTime timestamp static constraints = { val(blank:false) timestamp(blank:false) }   static belongsTo = [Instrument, Field]   String toString() { "${val} ${timestamp}" } }

Final scaffolding

Now we need to run

1 2 3 4 5

grails create-controller Datasource grails create-controller Field grails create-controller FieldValue grails create-controller Instrument grails create-controller InstrumentType

to create a controller for each domain class. Then put def scaffold = true in each controller to have Grails create basic CRUD functionality.



Conclusion

Grails is really easy to use and definitely more fun than the Java alternatives. Whenever I have time, I will retrieve some data and populate the database with some values. The next step would be to do some analysis on the data.



]]> http://ivanidris.net/wordpress/index.php/2010/07/11/grails-finance-0-1/feed 0 http://ivanidris.net/wordpress/index.php/2010/07/04/short-guide-to-scala http://ivanidris.net/wordpress/index.php/2010/07/04/short-guide-to-scala#comments Sun, 04 Jul 2010 09:57:05 +0000 admin http://ivanidris.net/wordpress/index.php/2010/07/04/short-guide-to-scala Dedicated to Scala Johansson




What is Scala?

Scala is a scalable language based on Java. It adds functional programming features to the normal Java OO. One of the main advantages of Scala is that you can write shorter programs than in Java and still benefit from the many Java libraries out there. For instance this code in Java

1 2 3 4 5 6 7

class AClass { private String myString;   public AClass(String string) { this.myString = string; } }

becomes

1	class AClass(myString: String)

As you can see semicolons are optional. Strangely enough Scala is statically typed. In fact the type system is pretty complex. I guess it is a matter of opinion whether this is good or bad.

Getting started

Variables

There are two types of variables that you can define val and var. A val represents a constant value and is final. A var can change.

1	val greeting = "Hi"

Notice that you don’t have to specify the type. This is called type inference, because greeting was initialized with a String literal, Scala figured out that greeting is of type String.

Functions

Functions are defined with def.

1 2 3 4

def min(i: Int, j: Int): Int = { if (i < j) x else j }

This code is pretty self explanatory. All the Java types have an equivalent in Scala with similar name. The exception is void which matches Unit in Scala.

Tuples

Scala adds tupples to the Java collection classes. Tuples can contain different type of elements. In Java you would have to create a holder class for this data. The code below creates a tuple and prints its contents

1 2 3

val myTuple = (1, "Ivan") println(myTuple._1) println(myTuple._2)

Simple scripts

Here is a script that can display lines in a file or files like the cat command in Unix.

1 2 3 4 5 6 7 8 9 10 11 12

#!/bin/sh exec scala "$0" "$@" !#   import scala.io.Source   if (args.length > 0) { for(arg <- args) { for (line <- Source.fromFile(arg).getLines) print(line) } }

You can run this as a shell script.

Pure OO

Scala classes cannot have static members. There are no primitives. Every value is an object. In Scala every method is an operator and vice versa. For instance we can define addition, multiplication and other operations for complex numbers.

1 2 3 4 5 6 7 8 9 10 11 12

class Complex(r : Double, i : Double) { var real = r val imaginary = i val modulus = Math.sqrt(r * r + i * i) def +(that: Complex):Complex = new Complex(real + that.real, imaginary + that.imaginary) def -(that: Complex):Complex = new Complex(real - that.real, imaginary - that.imaginary) def * (that: Complex):Complex = new Complex(real * that.real - imaginary * that.imaginary, real * that.imaginary + imaginary * that.real) }

and we can test it with JUnit or any other testing frameworks which is written in Java or Scala.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

import org.junit._   class ComplexSuite { val one = new Complex(1.0, 0) val two = new Complex(2.0, 0)   @Test def testAddition() { val sum = one + two Assert.assertEquals(sum.real, 3.0, 0.1) Assert.assertEquals(sum.imaginary, 0, 0.1) }   @Test def testSubtraction() { val difference = one - one Assert.assertEquals(difference.real, 0, 0.1) Assert.assertEquals(difference.imaginary, 0, 0.1) }   @Test def testMultiplication() { val product = one * two Assert.assertEquals(product.real, 2.0, 0.1) Assert.assertEquals(product.imaginary, 0, 0.1) }   @Test def testModulus() { Assert.assertEquals(one.modulus, 1.0, 0.1) Assert.assertEquals(two.modulus, 2.0, 0.1) } }

Complex numbers have many applications – one of the them is fractals like the Mandelbrot set. These type of fractals are defined by a recursive formula, where you calculate the next complex number in a series by multiplying the current complex number you have by itself and adding a constant to it. If z is a complex number, you have the relation z * z + c, where c is a constant complex number. This can be graphed in the complex plane with horizontal real values axis and vertical imaginary values axis. I used the so called “escape time algorithm” to draw the fractal. The algorithm scans the points in a small region around the origin on a distance of about 2. Actually I used a square which was probably a little bit too big. Each of these points is used as the c value and is assigned a color based on the number of iterations it takes to escape the region. If it takes more than a predefined number of iterations to escape the pixel gets the default background color.



Here is the Maven pom.xml

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79

<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion>   <groupId>ComplexNumbers</groupId> <artifactId>ComplexNumbers</artifactId> <version>1.0</version> <repositories> <repository> <id>scala-tools.org</id> <name>Scala-tools Maven2 Repository</name> <url>http://scala-tools.org/repo-releases</url> </repository> </repositories> <pluginRepositories> <pluginRepository> <id>scala-tools.org</id> <name>Scala-tools Maven2 Repository</name> <url>http://scala-tools.org/repo-releases</url> </pluginRepository> </pluginRepositories> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.7</version> </dependency> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>2.8.0.RC3</version> </dependency> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-swing</artifactId> <version>2.8.0.RC3</version> </dependency> </dependencies> <build> <pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>2.0.2</version> </plugin> <plugin> <groupId>org.scala-tools</groupId> <artifactId>maven-scala-plugin</artifactId> <version>2.9.1</version> </plugin> </plugins> </pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <executions> <execution> <phase>compile</phase> <goals> <goal>compile</goal> </goals> </execution> </executions> <configuration> <source>1.6</source> <target>1.6</target> </configuration> </plugin> <plugin> <groupId>org.scala-tools</groupId> <artifactId>maven-scala-plugin</artifactId> </plugin> </plugins> </build> </project>

We can extend the Mandelbrot idea and use other integer powers instead of 2. The resulting fractals are called the Multibrot set. This calls for an ** operator and also division for negative powers.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

def / (that: Complex):Complex = { val denominator: Double = Math.pow(that.modulus, 2) val realNumerator = real * that.real + imaginary * that.imaginary val imagNumerator = imaginary * that.real - real * that.imaginary   return new Complex( realNumerator/ denominator, imagNumerator/ denominator) }   def ** (n:Int):Complex = { var z = new Complex(r, i)   if(n > 0) { for(j <- 1 until n) { z *= new Complex(r, i) } } else { for(j <- n until 0) { z /= new Complex(r, i) } }   return z   }

As a side note the Moivre formula did not work properly although it should in theory. Since this is not production code I did not look further for better algorithms. The corresponding unit tests are here.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

@Test def testDivision() { Assert.assertEquals(2, two / one real, 0.1) Assert.assertEquals(0, two / one imaginary, 0.1) }   @Test def testPower() { val i = new Complex(0.0, 1.0) val i2 = i ** 2 Assert.assertEquals(i2.real, -1, 0.1) Assert.assertEquals(i2.imaginary, 0, 0.1)   Assert.assertEquals(1, one ** 2 real, 0.1) Assert.assertEquals(0, one ** 2 imaginary, 0.1)   Assert.assertEquals(4, two ** 2 real, 0.1) Assert.assertEquals(0, two ** 2 imaginary, 0.1)   Assert.assertEquals(8, two ** 3 real, 0.1) Assert.assertEquals(0, two ** 3 imaginary, 0.1) }

I also changed the main application class a bit. Scala has lots of wrappers for Swing, with lots of improvements. Unfortunately, sometimes it gets confusing because the names of the wrapper classes are often the same as the ones from swing.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

import java.awt.image.BufferedImage import java.awt.{Toolkit, Color, Graphics} import java.util.{Arrays, TimerTask, Timer} import scala.swing._ import event.ActionEvent   object FractalApp extends SimpleSwingApplication { val DIM = 400 val MAX_ITERATION = 300     def top = new MainFrame { title = "Fractal App" preferredSize_=(new Dimension(DIM, DIM)) contents = new Panel { var img = new BufferedImage(DIM,DIM,BufferedImage.TYPE_INT_RGB) this.peer.setDoubleBuffered(true) val timer = new Timer(); timer.schedule(new ScheduleTask(), 500);     override def paintComponent(g : Graphics2D) : Unit = { super.paintComponent(g)   g.drawImage(img, null, 0, 0) Toolkit.getDefaultToolkit().sync   g.dispose   }   class ScheduleTask extends TimerTask {   def run():Unit ={   for(d <- -9 to 9) { img.flush for(i <- 1 until DIM; j <- 1 until DIM) { val z0 = new Complex(-2.0 + i * 4.0 / DIM , -2.0 + j * 4.0 / DIM) val color = calculateColor(z0, 0, z0, d) img.setRGB(i, j, color) }   System.out.println(d) repaint(); } } } } }   def calculateColor(z:Complex, n:Int, z0:Complex, d:Int):Int ={ if(n == MAX_ITERATION) { return Color.BLACK.getRGB }   if(z.modulus >= 2) { return n * n * MAX_ITERATION * MAX_ITERATION }   calculateColor(z ** d + z0, n + 1, z0, d) } }

Demo run for powers -10 to 10



Conclusion

Scala is a new JVM language that is worth looking into. It shows lots of promise and offers new possibilities such as functional programming. You can write short scripts with Scala, Swing applications or anything else that you could write in Java. Scala tries to be better then Java without being Java, however in a statically typed way unlike Ruby or Groovy. Scala is by no means mainstream currently, but it might be by the time Scala Johansson graduates from college, starts a career etcetera, etcetera …



]]> http://ivanidris.net/wordpress/index.php/2010/07/04/short-guide-to-scala/feed 1