<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Joseph,<br>
<br>
I'm making some data visualizations and despite of not having an
advice on conceptual design, I share part of the practical problem
of having to work with CSV values in a Smalltalk environment and
some times with a lot of records (my recent project works with 270k
of them). The visualization I did was documented broadly at [1], but
essentially I create a "PublishedMedInfo class >>
loadDataFromCSV: aFile usingDelimiter: aCharacter" method that fill
out my domain objects that came from an excel (and then CSV) file.<br>
<br>
[1] <a class="moz-txt-link-freetext" href="http://mutabit.com/offray/blog/en/entry/sdv-infomed">http://mutabit.com/offray/blog/en/entry/sdv-infomed</a><br>
<br>
For my recent project [2] I'm using a SQLite bridge between Pharo
and the imported data from CVS. In that way I'm delegating storage
and querying (including duplicates) to a small but potent database
back-end, while using objects to model "higher" concerns of my
domain. I know some worries about objects-database mismatch
impedance, but working with data and its visualization/reporting
lets you to build bridges leveraging the former to the database and
the last to objects, while using the strengths of each one in their
own place.<br>
<br>
[2] <a class="moz-txt-link-freetext" href="https://twitter.com/offrayLC/status/725314838696701957">https://twitter.com/offrayLC/status/725314838696701957</a><br>
<br>
So my practical advice is to explore this kinds of combination early
in your design. May be a quick hands on mockup could let you know if
it works for you. In my case it has and I'm implementing it sooner
in my projects.<br>
<br>
Cheers,<br>
<br>
Offray<br>
<br>
Ps: Long time without writing, but I have been reading constantly.
Nice to be "back" :-)<br>
<br>
<div class="moz-cite-prefix">On 29/04/16 09:28, Joseph Alotta wrote:<br>
</div>
<blockquote
cite="mid:AC4066A1-5767-4B79-9BEB-086AD40A0EF6@gmail.com"
type="cite">Thanks for all the help.
<br>
<br>
I like the idea of having the code sense the format of the data
and acting accordingly.
<br>
<br>
For separators, I could count the number of each kind of
separators in the file and compare it to the number of lines. Say
3 or more separators per line.
<br>
<br>
Then I can parse by columns and look for the dominant data type.
For a column that is 60% matching a date type, I can assume it is
a date column and the mismatches are headers.
<br>
<br>
The amount should be numeric.
<br>
<br>
The payee should be mostly letters, etc.
<br>
<br>
One issue I have is knowing what to call the object that does
this. It would not be a Transaction, because this is a function
of many Transactions.
<br>
<br>
FileLoader? FileAnalyzer?
<br>
<br>
Also, at this point I should be looking for missing dates and
duplicates.
<br>
<br>
Duplicates are troublesome, since everytime I download the file,
it starts from the beginning of the year again. I keep
downloading them because I think they will only keep data for 6
months or so.
<br>
<br>
Also duplicate transactions are valid. Suppose I go into a coffee
shop and buy a cup of coffee, then go back the same day, same
store for a refill.
<br>
<br>
Your thoughts?
<br>
<br>
Sincerely,
<br>
<br>
Joe.
<br>
<br>
<br>
<br>
<hr align="left" width="300">
View this message in context: <a moz-do-not-send="true"
href="http://forum.world.st/conceptual-design-help-tp4892763p4892966.html">Re:
conceptual design help</a><br>
Sent from the <a moz-do-not-send="true"
href="http://forum.world.st/Squeak-Beginners-f107673.html">Squeak
- Beginners mailing list archive</a> at Nabble.com.<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Beginners mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Beginners@lists.squeakfoundation.org">Beginners@lists.squeakfoundation.org</a>
<a class="moz-txt-link-freetext" href="http://lists.squeakfoundation.org/mailman/listinfo/beginners">http://lists.squeakfoundation.org/mailman/listinfo/beginners</a>
</pre>
</blockquote>
<br>
</body>
</html>