Hello,
I am porting the FIX protocol to Squeak/Pharo. http://www.fixprotocol.org/
It is the Financial Information eXchange protocol. It is used by many firms in the financial industries, stocks, forex, etc.
There is good documentation and also a semi-reference implementation in Java. http://www.quickfixj.org
I have a partial port started but I wanted a check with those who know more than I about proper Smalltalk coding.
In the Java implementation there are over 2000 classes. In the protocol there are approximately 90 message types and 956 field types.
I am initially creating the message classes as they form the basis of the protocol and the fields are simply instance variables and methods in the message classes.
I am not porting the Java library. I am implementing the protocol in Smalltalk from the documentation.
Initially I have created a Fix44Field class with its necessary variables. There are several versions of the Protocol and FIX 4.4 is the one required for my application.
With this Fix44Field class I am creating a Dictionary with the 956 different field types. I am planning on creating a constructor method which will create the desired field from the information in the dictionary about the field and its properties.
Am I way off base here? Should I really create the 956 classes necessary to for each field in addition to the 90 or so message classes, and then I don't know what else I haven't discovered yet. Having 1000+ classes just seems unwieldy and naively it just doesn't feel like the Smalltalk way. I could be wrong, but I haven't seen such an example to my memory. I don't know enough about Java to know if it is good Java form either.
When I have a reasonably fleshed out, tested and working solution, I will make the source available under a MIT license. I would love to see Squeak/Pharo become an out of the box viable option for trading applications.
Advise and wisdom greatly appreciated.
Thanks.
Jimmie
On Fri, Dec 31, 2010 at 5:24 PM, Jimmie Houchin jlhouchin@gmail.com wrote:
In the Java implementation there are over 2000 classes. In the protocol there are approximately 90 message types and 956 field types.
[snip]
Am I way off base here? Should I really create the 956 classes necessary to for each field in addition to the 90 or so message classes, and then I don't know what else I haven't discovered yet. Having 1000+ classes just seems unwieldy and naively it just doesn't feel like the Smalltalk way. I could be wrong, but I haven't seen such an example to my memory. I don't know enough about Java to know if it is good Java form either.
[snip]
I'm not a Java expert, but I can sort of see the rational for having so many classes in a statically-typed language. You can use the type system to enforce the correctness of the protocol implementation. With a class for each type of field, you can declare variables with the types of fields expected by the protocol, and the compiler will tell you if you try to do something that doesn't fit the protocol specification.
In a dynamic language like Smalltalk, though, I'd expect classes that map to the *structure* of the protocol rather than it's specification. Perhaps many of those 956 field types are quite similar and can be modelled by just a handful of classes.
Another thing to look out for is opportunities to model things with composition instead of inheritance. The Java implementation is modelling the protocol largely in the type system, and as such it has to have single objects to which the types can be applied. Since you're going to be modelling the protocol with objects rather than types you're free to represent the elements of the protocol with clusters of objects. (For example, you might have a Field object that only represents the data in the field, and contains a reference to a FieldType object that describes the semantics of the field).
BTW, Congratulations on taking on such a big project - good to see it happening in Smalltalk.
Hope this helps,
Colin
On 12/31/2010 5:12 PM, Colin Putney wrote:
On Fri, Dec 31, 2010 at 5:24 PM, Jimmie Houchinjlhouchin@gmail.com wrote:
In the Java implementation there are over 2000 classes. In the protocol there are approximately 90 message types and 956 field types.
[snip]
Am I way off base here? Should I really create the 956 classes necessary to for each field in addition to the 90 or so message classes, and then I don't know what else I haven't discovered yet. Having 1000+ classes just seems unwieldy and naively it just doesn't feel like the Smalltalk way. I could be wrong, but I haven't seen such an example to my memory. I don't know enough about Java to know if it is good Java form either.
[snip] I'm not a Java expert, but I can sort of see the rational for having so many classes in a statically-typed language. You can use the type system to enforce the correctness of the protocol implementation. With a class for each type of field, you can declare variables with the types of fields expected by the protocol, and the compiler will tell you if you try to do something that doesn't fit the protocol specification.
In a dynamic language like Smalltalk, though, I'd expect classes that map to the *structure* of the protocol rather than it's specification. Perhaps many of those 956 field types are quite similar and can be modelled by just a handful of classes.
Another thing to look out for is opportunities to model things with composition instead of inheritance. The Java implementation is modelling the protocol largely in the type system, and as such it has to have single objects to which the types can be applied. Since you're going to be modelling the protocol with objects rather than types you're free to represent the elements of the protocol with clusters of objects. (For example, you might have a Field object that only represents the data in the field, and contains a reference to a FieldType object that describes the semantics of the field).
BTW, Congratulations on taking on such a big project - good to see it happening in Smalltalk.
Hope this helps,
Colin
Hello Colin,
Thanks for the advise.
I hadn't planned on following the Java model unless specifically advised to here. I understand that Java and Smalltalk are very different beasts. But I did want to double check and inquire.
The protocol is a purely message based protocol, where you have messages with specific and specified fields in those messages. The messages are sent in text from client to server. This is why my focus has been on the messages and the requirements of the messages. Provided that the messages are well formed, the creation and verification of the fields making up the messages isn't as important that they be their own independent classes.
Most of the fields are simply strings, chars (single character strings) or ints. Some have a set of values from which the value is required to be from. I have created my Fix44Field class to be able to handle the values, types and the constraints on the values without requiring a class for each type.
This allows me to easily do a couple of roundtrips on the documentation and have the fields implemented. The more I can write methods to parse the docs and create the classes the happier I am. :)
This will also help me to be able to easily update to newer versions. I really wanted to make sure I wasn't making any radical mistakes.
Doing this protocol helps me get away from having to interface a COM library or use a Java library. It will also allow me to migrate away from Windows in the future. :)
Hopefully as I implement a viable tool to implement trading strategies, it becomes an attractive option for people who want to create their own strategies but not have to program the entire application. It would be nice to have a small community grow around this over time. It is my privilege to contribute something back to the community I receive so much from.
Thanks for your help.
Jimmie
Hello Jimmie,
I had to do something similar a while ago. I had to support a complex XML schema for a standard message format of the Dutch government (StUF).
The Java implementation used JAXB to generate a few thousand data objects. Very clumsy.
In Smalltalk I made some code to generate Magritte descriptions for all the structure and field definitions in the XML schema. Using the Magritte descriptions it was very easy to read and write messages in the correct format. I created real classes only for the few message types that required some logic. For all the other message types I used some generic class that stores the values in a dictionary.
So my advice is to have a look at Magritte. It can help you handle the type information without creating/generating lots of classes.
Jan.
On Fri, Dec 31, 2010 at 11:24 PM, Jimmie Houchin jlhouchin@gmail.comwrote:
Hello,
I am porting the FIX protocol to Squeak/Pharo. http://www.fixprotocol.org/
It is the Financial Information eXchange protocol. It is used by many firms in the financial industries, stocks, forex, etc.
There is good documentation and also a semi-reference implementation in Java. http://www.quickfixj.org
I have a partial port started but I wanted a check with those who know more than I about proper Smalltalk coding.
In the Java implementation there are over 2000 classes. In the protocol there are approximately 90 message types and 956 field types.
I am initially creating the message classes as they form the basis of the protocol and the fields are simply instance variables and methods in the message classes.
I am not porting the Java library. I am implementing the protocol in Smalltalk from the documentation.
Initially I have created a Fix44Field class with its necessary variables. There are several versions of the Protocol and FIX 4.4 is the one required for my application.
With this Fix44Field class I am creating a Dictionary with the 956 different field types. I am planning on creating a constructor method which will create the desired field from the information in the dictionary about the field and its properties.
Am I way off base here? Should I really create the 956 classes necessary to for each field in addition to the 90 or so message classes, and then I don't know what else I haven't discovered yet. Having 1000+ classes just seems unwieldy and naively it just doesn't feel like the Smalltalk way. I could be wrong, but I haven't seen such an example to my memory. I don't know enough about Java to know if it is good Java form either.
When I have a reasonably fleshed out, tested and working solution, I will make the source available under a MIT license. I would love to see Squeak/Pharo become an out of the box viable option for trading applications.
Advise and wisdom greatly appreciated.
Thanks.
Jimmie
On 31/12/10 5:24 PM, Jimmie Houchin wrote:
I am initially creating the message classes as they form the basis of the protocol and the fields are simply instance variables and methods in the message classes.
I had a look at:
http://en.wikipedia.org/wiki/Financial_Information_eXchange http://www.quickfixengine.org/FIX44.html http://www.quickfixengine.org/xml.html
In a lot of cases, many fields are optional, so I doubt an instance variable per message field is the right thing to do.
I am not porting the Java library. I am implementing the protocol in Smalltalk from the documentation.
I think a lot of the code can be generated from the xml file corresponding to the FIX44.html above.
What code should be generated is the big question. You could generate classes for each message type and each field type, and get the code bloat you're concerned about. Or, you could generate data structures from the spec, that are used to construct/parse messages and fields. If you generate all this stuff then you can delay the choice.
Initially I have created a Fix44Field class with its necessary variables. There are several versions of the Protocol and FIX 4.4 is the one required for my application.
With this Fix44Field class I am creating a Dictionary with the 956 different field types. I am planning on creating a constructor method which will create the desired field from the information in the dictionary about the field and its properties.
Am I way off base here? Should I really create the 956 classes necessary to for each field in addition to the 90 or so message classes, and then I don't know what else I haven't discovered yet. Having 1000+ classes just seems unwieldy and naively it just doesn't feel like the Smalltalk way. I could be wrong, but I haven't seen such an example to my memory. I don't know enough about Java to know if it is good Java form either.
IMHO, you'll end up with a class for each of the 90 or so message types, due to validation and constraint needs. Whereas, the 1000+ field types will be handled generically by a few classes.
Good luck.
On 1/1/2011 12:29 PM, Yanni Chiu wrote:
On 31/12/10 5:24 PM, Jimmie Houchin wrote:
I am initially creating the message classes as they form the basis of the protocol and the fields are simply instance variables and methods in the message classes.
I had a look at:
http://en.wikipedia.org/wiki/Financial_Information_eXchange http://www.quickfixengine.org/FIX44.html http://www.quickfixengine.org/xml.html
In a lot of cases, many fields are optional, so I doubt an instance variable per message field is the right thing to do.
I am not porting the Java library. I am implementing the protocol in Smalltalk from the documentation.
I think a lot of the code can be generated from the xml file corresponding to the FIX44.html above.
What code should be generated is the big question. You could generate classes for each message type and each field type, and get the code bloat you're concerned about. Or, you could generate data structures from the spec, that are used to construct/parse messages and fields. If you generate all this stuff then you can delay the choice.
Initially I have created a Fix44Field class with its necessary variables. There are several versions of the Protocol and FIX 4.4 is the one required for my application.
With this Fix44Field class I am creating a Dictionary with the 956 different field types. I am planning on creating a constructor method which will create the desired field from the information in the dictionary about the field and its properties.
Am I way off base here? Should I really create the 956 classes necessary to for each field in addition to the 90 or so message classes, and then I don't know what else I haven't discovered yet. Having 1000+ classes just seems unwieldy and naively it just doesn't feel like the Smalltalk way. I could be wrong, but I haven't seen such an example to my memory. I don't know enough about Java to know if it is good Java form either.
IMHO, you'll end up with a class for each of the 90 or so message types, due to validation and constraint needs. Whereas, the 1000+ field types will be handled generically by a few classes.
Good luck.
Hello Yanni,
Thanks.
I fully expected to create classes for all of the message types. That seemed reasonable and the correct way to do it. But the field types in general I thought could be generated from some sort of template with the exception of a few which might be more complicated and require their own class.
Thanks for the reply.
Jimmie
Cool Jimmie! Hey, about 6 years ago I did an implementation of the FIX protocol; v41 and 42 IIRC. I is incomplete but I think it was able to pass the logon and heartbeat test at... that public FIX protocol testing facility (sorry, can't remember where it was at the moment).
It handles the FIX async protocol; sequence-numbers, gap-filling, etc. It was originally going to be for a pay job, but unfortunately the job didn't work out and so I abandoned the work.
But the work done to that point spans nearly a year and may provide a good starting point on which you could finish it or at least get ideas from it. I haven't been in the code since 2005, but I considered it a good design.
I've created a new project on Squeaksource called "Fix Protocol" with you and I as administrators.
I hope you find it useful. I'm really glad you are doing this which I was never able to get back and finish!
Regards, Chris
On Fri, Dec 31, 2010 at 4:24 PM, Jimmie Houchin jlhouchin@gmail.com wrote:
Hello,
I am porting the FIX protocol to Squeak/Pharo. http://www.fixprotocol.org/
It is the Financial Information eXchange protocol. It is used by many firms in the financial industries, stocks, forex, etc.
There is good documentation and also a semi-reference implementation in Java. http://www.quickfixj.org
I have a partial port started but I wanted a check with those who know more than I about proper Smalltalk coding.
In the Java implementation there are over 2000 classes. In the protocol there are approximately 90 message types and 956 field types.
I am initially creating the message classes as they form the basis of the protocol and the fields are simply instance variables and methods in the message classes.
I am not porting the Java library. I am implementing the protocol in Smalltalk from the documentation.
Initially I have created a Fix44Field class with its necessary variables. There are several versions of the Protocol and FIX 4.4 is the one required for my application.
With this Fix44Field class I am creating a Dictionary with the 956 different field types. I am planning on creating a constructor method which will create the desired field from the information in the dictionary about the field and its properties.
Am I way off base here? Should I really create the 956 classes necessary to for each field in addition to the 90 or so message classes, and then I don't know what else I haven't discovered yet. Having 1000+ classes just seems unwieldy and naively it just doesn't feel like the Smalltalk way. I could be wrong, but I haven't seen such an example to my memory. I don't know enough about Java to know if it is good Java form either.
When I have a reasonably fleshed out, tested and working solution, I will make the source available under a MIT license. I would love to see Squeak/Pharo become an out of the box viable option for trading applications.
Advise and wisdom greatly appreciated.
Thanks.
Jimmie
able to pass the logon and heartbeat test at... that public FIX protocol testing facility (sorry, can't remember where it was at the moment).
Ah, it was this one:
Hello Chris,
Thanks for the reply.
I'll check out your code see what I can learn and proceed from there. I'm sure there will plenty to use and learn from. Thanks for creating the project.
Since this is for my personal project, I most definitely will persist until complete enough for use. I'm not sure about the FIXML stuff. I am not opposed to it, it just isn't a priority for me at this time, and isn't required for my trading. However, I would like it to become a full FIX Protocol engine at some point in time.
On 1/1/2011 3:12 PM, Chris Muller wrote:
http://www.fixprotocol.org/products/detail/4649
Thanks for this link. I wasn't aware of this service. That will be a valuable resource.
I am glad my post brought to surface unknown code. It will be a valuable resource.
Thanks again.
Jimmie
On 1/1/2011 3:06 PM, Chris Muller wrote:
Cool Jimmie! Hey, about 6 years ago I did an implementation of the FIX protocol; v41 and 42 IIRC. I is incomplete but I think it was able to pass the logon and heartbeat test at... that public FIX protocol testing facility (sorry, can't remember where it was at the moment).
It handles the FIX async protocol; sequence-numbers, gap-filling, etc. It was originally going to be for a pay job, but unfortunately the job didn't work out and so I abandoned the work.
But the work done to that point spans nearly a year and may provide a good starting point on which you could finish it or at least get ideas from it. I haven't been in the code since 2005, but I considered it a good design.
I've created a new project on Squeaksource called "Fix Protocol" with you and I as administrators.
I hope you find it useful. I'm really glad you are doing this which I was never able to get back and finish!
Regards, Chris
On Fri, Dec 31, 2010 at 4:24 PM, Jimmie Houchinjlhouchin@gmail.com wrote:
Hello,
I am porting the FIX protocol to Squeak/Pharo. http://www.fixprotocol.org/
It is the Financial Information eXchange protocol. It is used by many firms in the financial industries, stocks, forex, etc.
There is good documentation and also a semi-reference implementation in Java. http://www.quickfixj.org
I have a partial port started but I wanted a check with those who know more than I about proper Smalltalk coding.
In the Java implementation there are over 2000 classes. In the protocol there are approximately 90 message types and 956 field types.
I am initially creating the message classes as they form the basis of the protocol and the fields are simply instance variables and methods in the message classes.
I am not porting the Java library. I am implementing the protocol in Smalltalk from the documentation.
Initially I have created a Fix44Field class with its necessary variables. There are several versions of the Protocol and FIX 4.4 is the one required for my application.
With this Fix44Field class I am creating a Dictionary with the 956 different field types. I am planning on creating a constructor method which will create the desired field from the information in the dictionary about the field and its properties.
Am I way off base here? Should I really create the 956 classes necessary to for each field in addition to the 90 or so message classes, and then I don't know what else I haven't discovered yet. Having 1000+ classes just seems unwieldy and naively it just doesn't feel like the Smalltalk way. I could be wrong, but I haven't seen such an example to my memory. I don't know enough about Java to know if it is good Java form either.
When I have a reasonably fleshed out, tested and working solution, I will make the source available under a MIT license. I would love to see Squeak/Pharo become an out of the box viable option for trading applications.
Advise and wisdom greatly appreciated.
Thanks.
Jimmie
Hi Jimmie,
I implemented part of the FIX protocol as part of one of my projects a while ago. I made very few classes. The fields mostly map to simple Smalltalk classes like strings, ints, decimals, data and times. Some fields are compound fields in that they contain other fields and are repeated in groups.
I did not make a class for each FIX message. I used dictionaries to map what fields were expected in each FIX message. I mapped the FIX number based field names to symbols that I made up that made sense to me and matched the FIX name for the field. This made for fairly simple code to parse/print FIX messages to and from my FIX message class. The data is stored in dictionaries with my symbols as keys and the Smalltalk value of the FIX field as the value in the dictionary.
This may not seem like good Smalltalk practice but I found that I wasn't really interested in doing anything with the FIX classes other than reading in FIX messages (data) and sending out FIX messages. For working with the data, I had other classes the matched what the system was doing. This often involved getting/sending much of the same data to/from another system that had its own message format.
In that case I have basically the same data in a whole bunch of classes that match say a non FIX message format and an equal number of classes that match the FIX format. Which one should you use to work with the data within the system? So, on both sides I have simple classes that basically just move the field data to and from dictionaries by parsing or printing the values in the correct format.
This made for some less than pretty code like:
asc := msg at: #AccountSourceCode.
instead of:
asc := msg accountSourceCode.
but it allowed for moving large numbers of fields from one message format to another with a loop that ran a dictionary that had matching symbolic field names for each message format.
orderedCollectionOfFieldsToCopy do: [:f | | fixSymbol | fixSymbol := dictionaryOfCorrespondingFields at: f. fixMessage at: f put: (nonFixData at: fixSymbol). ].
Again, I don't know if this is in the best tradition of Smalltalk but in my multi message format environment, it seemed best.
Lou ----------------------------------------------------------- Louis LaBrunda Keystone Software Corp. SkypeMe callto://PhotonDemon mailto:Lou@Keystone-Software.com http://www.Keystone-Software.com
On 1/3/2011 10:05 AM, Louis LaBrunda wrote:
Hi Jimmie,
I implemented part of the FIX protocol as part of one of my projects a while ago. I made very few classes. The fields mostly map to simple Smalltalk classes like strings, ints, decimals, data and times. Some fields are compound fields in that they contain other fields and are repeated in groups.
I did not make a class for each FIX message. I used dictionaries to map what fields were expected in each FIX message. I mapped the FIX number based field names to symbols that I made up that made sense to me and matched the FIX name for the field. This made for fairly simple code to parse/print FIX messages to and from my FIX message class. The data is stored in dictionaries with my symbols as keys and the Smalltalk value of the FIX field as the value in the dictionary.
This may not seem like good Smalltalk practice but I found that I wasn't really interested in doing anything with the FIX classes other than reading in FIX messages (data) and sending out FIX messages. For working with the data, I had other classes the matched what the system was doing. This often involved getting/sending much of the same data to/from another system that had its own message format.
In that case I have basically the same data in a whole bunch of classes that match say a non FIX message format and an equal number of classes that match the FIX format. Which one should you use to work with the data within the system? So, on both sides I have simple classes that basically just move the field data to and from dictionaries by parsing or printing the values in the correct format.
This made for some less than pretty code like:
asc := msg at: #AccountSourceCode.
instead of:
asc := msg accountSourceCode.
but it allowed for moving large numbers of fields from one message format to another with a loop that ran a dictionary that had matching symbolic field names for each message format.
orderedCollectionOfFieldsToCopy do: [:f | | fixSymbol | fixSymbol := dictionaryOfCorrespondingFields at: f. fixMessage at: f put: (nonFixData at: fixSymbol). ].
Again, I don't know if this is in the best tradition of Smalltalk but in my multi message format environment, it seemed best.
Lou
Hello Lou,
I don't think any of that sounds unreasonable. I just knew I wasn't wanting to create 1100 classes for the field types and the message types. I have been looking at the field types in the documentation and it looked reasonable to do exactly what you said, and I had planned on doing similarly. But I hadn't quite gotten to looking at doing similarly for the message types. I will take a look at that. The more that I can automate, the faster I can get it working and spend my time on more interesting problems. And like you say it is easier to add or move to other versions. I know my current need is for version 4.4. But I don't what other brokers or other the future requirements will be. It is nice be agile.
I think this is somewhat the simplest thing that could possibly work. If performance is insufficient or any optimization is required, it could then be done on a message by message, or field by field basis and still maintain a working system.
Thanks for sharing your experience and advise.
Jimmie
Hi Jimmie,
I took a quick look at my code and thought I would share a little more with you. I created a class to hold some information about a FIX field's definition. It had these instance variables "fieldName tag inCvtBlock outCvtBlock isLength needsLengthTag fieldClassName" whose name are fairly indicative of what they hold.
I create an instance of this class for each field in a given FIX message and add it to a class variable dictionary keyed by the FIX tag number. When parsing the FIX message, I lookup the FIX field by its tag and the info in its corresponding instance tells me how to convert and save the value.
I also created a class with these instance variables "tag value fieldDef". When parsing FIX messages, I save an instance of this class in a dictionary, instead of just the value. This way I can more easily create a FIX message for output from this information.
Lou ----------------------------------------------------------- Louis LaBrunda Keystone Software Corp. SkypeMe callto://PhotonDemon mailto:Lou@Keystone-Software.com http://www.Keystone-Software.com
squeak-dev@lists.squeakfoundation.org
-
Chris Muller -
Colin Putney -
Jan van de Sandt -
Jimmie Houchin -
Louis LaBrunda -
Yanni Chiu