[etoys-dev] Etoys Inbox: System-Richo.18.mcz
commits at source.squeak.org
commits at source.squeak.org
Tue May 25 00:59:53 EDT 2010
A new version of System was added to project Etoys Inbox:
http://source.squeak.org/etoysinbox/System-Richo.18.mcz
==================== Summary ====================
Name: System-Richo.18
Author: Richo
Time: 25 May 2010, 1:59:04 am
UUID: c21f1dfd-8a00-e648-b388-ec8cdeb92e3a
Ancestors: System-bf.16
* Fixed LocaleID>>#displayName
=============== Diff against System-bf.16 ===============
Item was added:
+ ----- Method: MOFile>>load1:localeID: (in category 'experimental') -----
+ load1: aFileName localeID: id
+ "CASE1:
+ all of strings are loaded.
+ translation strings are converted to Squeak format on load time.
+ original-string/index pairs are registerd to Dictionary on load time.
+ hash search can't be used"
+ | strm originalTable translatedTable |
+ localeID _ id.
+ strm_ FileStream readOnlyFileNamed: aFileName.
+ fileName _ aFileName.
+ [
+ self loadHeader: strm.
+ originalTable _ self loadStringPointers: strm
+ offset: originalTableOffset.
+
+ originalStrings _ self loadStrings: strm
+ pointers: originalTable.
+
+ translatedTable _ self loadStringPointers: strm
+ offset: translatedTableOffset.
+
+ translatedStrings _ self loadStrings: strm
+ pointers: translatedTable
+ encoding: 'utf8'
+ languageEnvironment: (Locale localeID: localeID) languageEnvironment .
+
+ translations _ Dictionary new.
+ 1 to: nStrings do: [:index |
+ | key |
+ key _ originalStrings at: index.
+ translations at: key put: index.
+ ].
+ originalTable _ nil.
+ ] ensure: [strm close].!
Item was added:
+ ----- Method: MOFile>>loadString:pointer:length: (in category 'private') -----
+ loadString: strm pointer: top length: len
+ | str |
+ str _ ByteString new: len.
+ strm position: top.
+ strm nextInto: str.
+ ^str replaceAll: Lf with: Cr.
+ !
Item was added:
+ ----- Method: MOFile>>translatedString: (in category 'private') -----
+ translatedString: index
+ "KNOWN PROBLEM: conversion is executed everytimes this method called"
+ | str |
+ str _ translatedStrings at: index.
+
+ ^str utf8ToSqueak applyLanguageInfomation: (Locale localeID: localeID) languageEnvironment.
+ !
Item was added:
+ ----- Method: MOFile>>loadStrings:pointers: (in category 'private') -----
+ loadStrings: strm pointers: table
+ ^self loadStrings: strm pointers: table encoding: nil languageEnvironment: nil
+ !
Item was added:
+ ----- Method: TextDomainManager classSide>>allMethodsWithTranslations (in category 'accessing') -----
+ allMethodsWithTranslations
+ "Look for #translated calls"
+ | methodsWithTranslations |
+ methodsWithTranslations := TranslatedReceiverFinder new stringReceiversWithContext: #translated.
+ methodsWithTranslations := methodsWithTranslations, (TranslatedReceiverFinder new
+ stringReceiversWithContext: #translatedNoop).
+
+ methodsWithTranslations := methodsWithTranslations collect: [:each | each key compiledMethod].
+
+ "Look for Etoys tiles and vocabularies"
+ methodsWithTranslations := methodsWithTranslations, (EToyVocabulary allPhrasesWithContextToTranslate collect: [:r |
+ (MethodReference new setStandardClass: r second methodSymbol: r third) compiledMethod.
+ ]).
+
+ ^methodsWithTranslations!
Item was added:
+ ----- Method: TextDomainManager classSide>>domainForPackage: (in category 'accessing') -----
+ domainForPackage: aPackageInfo
+ "Package names and text domains are synonyms now"
+ ^aPackageInfo name!
Item was added:
+ ----- Method: MOFile>>loadHeader: (in category 'private') -----
+ loadHeader: strm
+ strm binary.
+ magic _ strm uint32.
+ magic = 16rDE120495
+ ifTrue: [isLittleEndian _ true]
+ ifFalse: [
+ magic = 16r950412DE
+ ifTrue: [isLittleEndian _ false]
+ ifFalse: [ self error: 'invalid MO']
+ ].
+ revision _ self nextInt32From: strm.
+ nStrings _ self nextInt32From: strm.
+ originalTableOffset _ self nextInt32From: strm.
+ translatedTableOffset _ self nextInt32From: strm.
+ hashTableSize _ self nextInt32From: strm.
+ hashTableOffset _ self nextInt32From: strm.
+ !
Item was added:
+ ----- Method: MOFile>>searchByDictionary: (in category 'public') -----
+ searchByDictionary: aString
+ | index |
+ index _ translations at: aString ifAbsent: [^nil].
+ ^self translatedString: index
+
+ !
Item was added:
+ ----- Method: MOFile classSide>>initialize (in category 'class initialization') -----
+ initialize
+ Cr := Character cr.
+ Lf := Character lf.
+ !
Item was added:
+ ----- Method: MOFile>>hashPjw: (in category 'experimental') -----
+ hashPjw: aString
+ "So called `hashpjw' function by P.J. Weinberger
+ [see Aho/Sethi/Ullman, COMPILERS: Principles, Techniques and Tools,
+ 1986, 1987 Bell Telephone Laboratories, Inc.] "
+ | stringSize hash g |
+ stringSize _ aString size.
+ hash _ 0.
+ 1 to: stringSize do: [:pos |
+ hash _ hash bitShift: 4.
+ hash _ hash + ((aString at: pos) asInteger).
+ g _ hash bitAnd: 16rF0000000.
+ g = 0 ifFalse: [
+ hash _ hash bitXor: (g bitShift: -24).
+ hash _ hash bitXor: g.
+ ]
+ ].
+ ^hash.
+ !
Item was added:
+ Object subclass: #TextDomainManager
+ instanceVariableNames: ''
+ classVariableNames: 'ClassCategories Classes DefaultDomain DomainInfos LoneClasses Packages'
+ poolDictionaries: ''
+ category: 'System-Localization'!
+ TextDomainManager class
+ instanceVariableNames: 'defaultDomain'!
+
+ !TextDomainManager commentStamp: 'tk 1/4/2008 16:08' prior: 0!
+ I manages mapping from class category to textdomain.
+
+ Class variables:
+ ClassCategories IdentityDictionary -- classCategory -> domainName
+ Classes IdentityDictionary -- class name (a Symbol) -> domainName (a cache only!!)
+ DefaultDomain String -- the default domain name
+ DomainInfos Dictionary -- domainName -> a TextDomainInfo
+ LoneClasses IdentityDictionary -- class name (a Symbol) -> domainName. For classes whose entire category are not all in the same domain (BookMorph and QuickGuideMorph)
+
+ TextDomainManager registerCategoryPrefix: 'DrGeoII' domain: 'DrGeoII'.
+ TextDomainManager unregisterDomain: 'DrGeoII'.
+
+ TextDomainManager registerClass: #QuickGuideMorph domain: 'quickguides'.
+ TextDomainManager registerClass: #QuickGuideHolderMorph domain: 'quickguides'.
+ !
+ TextDomainManager class
+ instanceVariableNames: 'defaultDomain'!
Item was added:
+ ----- Method: MOFile>>load4:localeID: (in category 'experimental') -----
+ load4: aFileName localeID: id
+ "CASE4:
+ all of strings are loaded.
+ loading and conversion of translation strings to Squeak format is executed on initialization time.
+ only hash search can be used"
+ | strm originalTable translatedTable |
+ localeID _ id.
+ strm_ FileStream readOnlyFileNamed: aFileName.
+ fileName _ aFileName.
+ [
+ self loadHeader: strm.
+ self loadHashTable: strm.
+ originalTable _ self loadStringPointers: strm
+ offset: originalTableOffset.
+
+ originalStrings _ self loadStrings: strm
+ pointers: originalTable.
+
+ translatedTable _ self loadStringPointers: strm
+ offset: translatedTableOffset.
+
+ translatedStrings _ self loadStrings: strm
+ pointers: translatedTable
+ encoding: 'utf-8'
+ languageEnvironment: (Locale localeID: localeID) languageEnvironment .
+ ] ensure: [strm close].!
Item was added:
+ ----- Method: MOFile>>loadHashTable: (in category 'experimental') -----
+ loadHashTable: strm
+ | entry |
+ hashTable _ IntegerArray ofSize: hashTableSize.
+ strm binary.
+ strm position: hashTableOffset.
+ 1 to: hashTableSize do: [:index |
+ entry _ self nextInt32From: strm.
+ hashTable at: index put: entry.
+ ]!
Item was added:
+ ----- Method: TextDomainManager classSide>>domainOfMethod: (in category 'accessing') -----
+ domainOfMethod: aCompiledMethod
+ ^ aCompiledMethod
+ propertyValueAt: self textDomainProperty
+ ifAbsent: [self updateDomainOfMethod: aCompiledMethod] !
Item was added:
+ ----- Method: TextDomainManager classSide>>clearAllDomains (in category 'private') -----
+ clearAllDomains
+ SystemNavigation default
+ allCompiledMethodDo: [:each | each
+ removeProperty: self textDomainProperty
+ ifAbsent: []] !
Item was added:
+ ----- Method: MOFile>>nextInt32From: (in category 'private') -----
+ nextInt32From: strm
+ ^isLittleEndian
+ ifTrue: [^strm nextLittleEndianNumber: 4]
+ ifFalse: [^strm nextInt32]!
Item was added:
+ ----- Method: TextDomainManager classSide>>updateDomainOfAllMethodsWithTranslations (in category 'private') -----
+ updateDomainOfAllMethodsWithTranslations
+ self allMethodsWithTranslations do: [:each|
+ self updateDomainOfMethod: each
+ ]!
Item was added:
+ ----- Method: TextDomainManager classSide>>initialize (in category 'class initialization') -----
+ initialize
+ " TextDomainManager initialize "
+ self defaultDomain: 'Etoys'; clearAllDomains!
Item was added:
+ ----- Method: MOFile>>testSearchByHash (in category 'experimental') -----
+ testSearchByHash
+ InternalTranslator allKnownPhrases
+ do: [:each |
+ self searchByHash: each
+ ].
+ !
Item was added:
+ ----- Method: TextDomainManager classSide>>textDomainProperty (in category 'private') -----
+ textDomainProperty
+ ^#textDomain!
Item was added:
+ ----- Method: MOFile>>loadStringPointers:offset: (in category 'private') -----
+ loadStringPointers: strm offset: tableOffset
+ "returns tupple {arrayOfOffsetToString arrayOfLengthOfString}"
+ | offsetTable lenTable len offset tupple |
+ offsetTable _ IntegerArray new: nStrings.
+ lenTable _ IntegerArray new: nStrings.
+ strm binary.
+ strm position: tableOffset.
+ 1 to: nStrings do: [:index |
+ len _ self nextInt32From: strm.
+ offset _ self nextInt32From: strm.
+ offsetTable at: index put: offset.
+ lenTable at: index put: len.
+ ].
+ tupple _ Array new: 2.
+ tupple at: 1 put: offsetTable.
+ tupple at: 2 put: lenTable.
+ ^tupple
+ !
Item was added:
+ ----- Method: TextDomainManager classSide>>defaultDomain (in category 'accessing') -----
+ defaultDomain
+ "I'm not sure we still need a default domain, AFAIK the default domain will only be used when no domain is found. In that case, wouldn't it be better to just look for a translation in all domains?"
+ ^defaultDomain!
Item was added:
+ ----- Method: TextDomainManager classSide>>domainForClass: (in category 'accessing') -----
+ domainForClass: aClass
+ ^'etoys'!
Item was added:
+ ----- Method: MOFile classSide>>fileName:localeID: (in category 'instance creation') -----
+ fileName: path localeID: id
+ ^self new
+ load:path localeID: id!
Item was added:
+ ----- Method: TextDomainManager classSide>>updateDomainOfMethod: (in category 'private') -----
+ updateDomainOfMethod: aCompiledMethod
+ "First it looks for the package of the method reference (using
+ the PackageOrganizer: deadly slow). If the method doesn't
+ belong to any package it uses the default domain. Finally it
+ stores the text domain of the method using a method
+ property, this way we gain performance the next time we
+ translate the same method because we avoid the use of
+ PackageOrganizer. Have I mentioned it is really slow? :)"
+ | package |
+ package := PackageOrganizer default
+ packageOfMethod: aCompiledMethod methodReference
+ ifNone: [].
+ ^ aCompiledMethod
+ propertyValueAt: self textDomainProperty
+ put: (package isNil
+ ifTrue: [TextDomainManager defaultDomain]
+ ifFalse: [package name])!
Item was added:
+ Object subclass: #MOFile
+ instanceVariableNames: 'localeID fileName isLittleEndian magic revision nStrings originalTableOffset translatedTableOffset hashTableSize hashTableOffset hashTable originalStrings translatedStrings translations'
+ classVariableNames: 'Cr Lf'
+ poolDictionaries: ''
+ category: 'System-Localization'!
+
+ !MOFile commentStamp: '<historical>' prior: 0!
+ Wrapper for MO file of gettext.
+ Known limitation:
+ currently don't support prural form.
+ translation strings have to be encoded in utf-8.
+
+ Implementation notes:
+ Testing on XO showed emulation of hash search without plugin + on demand loading is slow.
+ The test also showed conversion of utf8 string to Squeak's String is really slow (especially for non-latin language).
+ so in this version, all of original/translated strings are loaded on initiaization,
+ but "translated strings" is left as ByteString on loading time, to reduce loading time.
+ After that the translated string is converted on demand.
+ !
Item was added:
+ ----- Method: MOFile>>searchByHash: (in category 'experimental') -----
+ searchByHash: aString
+ | hashValue nstr index incr key |
+ hashValue _ self hashPjw: aString.
+ incr _ 1 + (hashValue \\ (hashTableSize -2)).
+ index _ (hashValue \\ hashTableSize) .
+ [ nstr _ (hashTable at: index +1 ).
+ nstr = 0 ifTrue: [^nil].
+ key _ self originalString: nstr.
+ key = aString ifTrue: [^self translatedString: nstr].
+ index >= (hashTableSize - incr)
+ ifTrue: [index _ index - (hashTableSize - incr) ]
+ ifFalse:[index _ index + incr].
+ ] doWhileTrue: true.!
Item was added:
+ ----- Method: MOFile>>translateByHash: (in category 'experimental') -----
+ translateByHash: aString
+ | trans |
+ trans _ self searchByHash: aString.
+ trans isNil ifTrue: [^aString]
+ ifFalse: [^trans].
+ !
Item was added:
+ ----- Method: MOFile>>originalString: (in category 'private') -----
+ originalString: index
+ ^originalStrings at: index.
+ !
Item was added:
+ ----- Method: MOFile>>loadStrings:pointers:encoding:languageEnvironment: (in category 'private') -----
+ loadStrings: strm pointers: tupple encoding: encodingName languageEnvironment: env
+ | strings rawStr str offsetTable lenTable |
+ offsetTable _ tupple first.
+ lenTable _ tupple second.
+ strings _ Array new: nStrings.
+ 1 to: nStrings do: [:index |
+ rawStr _ self loadString: strm
+ pointer: (offsetTable at: index)
+ length: (lenTable at: index).
+ str _ encodingName isNil ifTrue: [rawStr]
+ ifFalse: [ encodingName = 'utf8'
+ ifTrue: [rawStr utf8ToSqueak applyLanguageInfomation: env]
+ ifFalse: [self error: 'this encoding isn''t supported']
+ ].
+ strings at: index put: str.
+ ].
+ ^strings.!
Item was added:
+ ----- Method: MOFile>>testSearchByDictionary (in category 'experimental') -----
+ testSearchByDictionary
+ InternalTranslator allKnownPhrases
+ do: [:each |
+ self searchByDictionary: each
+ ].
+ !
Item was added:
+ ----- Method: MOFile>>fileName: (in category 'public') -----
+ fileName: path
+ fileName _ path!
Item was added:
+ ----- Method: TextDomainManager classSide>>allKnownDomains (in category 'accessing') -----
+ allKnownDomains
+ "Every package has it's own text domain now so it's not necessary to keep a registry of all domains, we can simply return all the packages in the image.
+ PROBLEM: If a package doesn't contain translations, it won't have a mo file but the GetTextTranslator will try to load it anyway. This happens when we switch languages. So far I tested it briefly and it seems to work..."
+ ^PackageOrganizer default packageNames!
Item was added:
+ ----- Method: MOFile>>atRandom (in category 'public') -----
+ atRandom
+
+ ^ self translatedString:nStrings atRandom.
+ !
Item was added:
+ ----- Method: MOFile>>translationFor: (in category 'public') -----
+ translationFor: aString
+ | file |
+ aString size = 0 ifTrue: [^ '']. "Gettext header"
+ ^ (self searchByDictionary: aString) ifNil: [
+ file := (fileName subStrings: '\').
+ Transcript show: aString printString, ' not found in ', "(file atLast: 3),'\', "(file last); cr.
+ ^aString]
+ !
Item was added:
+ ----- Method: TextDomainManager classSide>>defaultDomain: (in category 'accessing') -----
+ defaultDomain: aDomainName
+ defaultDomain := aDomainName!
Item was changed:
----- Method: LocaleID>>displayName (in category 'accessing') -----
displayName
"Answer a proper name to represent the receiver in GUI.
The wording is provided by translations of the magic value
'<language display name>'.
'English' -> 'English'
'German' -> 'Deutsch'
"
| magicPhrase translatedMagicPhrase |
+ magicPhrase := 'Language-Name'.
- magicPhrase := '<language display name>'.
translatedMagicPhrase := NaturalLanguageTranslator translateWithoutLoading: magicPhrase toLocaleID: self.
^ translatedMagicPhrase = magicPhrase
ifTrue: [self displayLanguage]
ifFalse: [translatedMagicPhrase]!
Item was added:
+ ----- Method: MOFile>>load:localeID: (in category 'public') -----
+ load: aFileName localeID: id
+ "all of original/translated strings are loaded.
+ but conversion of translation string (in utf-8 bytestring) to Squeak format will be defered.
+ original-string/index pairs are registerd to Dictionary on load time.
+ hash search can't be used"
+ | strm originalTable translatedTable |
+ localeID _ id.
+ strm_ FileStream readOnlyFileNamed: aFileName.
+ fileName _ aFileName.
+ [
+ self loadHeader: strm.
+ originalTable _ self loadStringPointers: strm
+ offset: originalTableOffset.
+
+ originalStrings _ self loadStrings: strm
+ pointers: originalTable.
+
+ translatedTable _ self loadStringPointers: strm
+ offset: translatedTableOffset.
+
+ translatedStrings _ self loadStrings: strm
+ pointers: translatedTable.
+
+ translations _ Dictionary new: nStrings * 2. "make too enough room to avoid #grow"
+ 1 to: nStrings do: [:index |
+ | key |
+ key _ originalStrings at: index.
+ translations at: key put: index.
+ ].
+ originalStrings _ nil.
+ ] ensure: [strm close].!
Item was added:
+ ----- Method: MOFile>>fileName (in category 'public') -----
+ fileName
+ ^fileName!
More information about the etoys-dev
mailing list