Hi -
I was just trying to implement the code for condensing the changes to the end of the sources file and ran into some 'interesting' issues. I would expect this to work:
fullName := 'test1.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. FileDirectory default copyFile: sourcesFile toFile: newFile. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
But it fails. Then I tried something else:
fullName := 'test2.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. [sourcesFile atEnd] whileFalse:[newFile nextChunkPut: sourcesFile nextChunk]. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
That fails to. Then I tried:
fullName := 'test3.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. [sourcesFile atEnd] whileFalse:[newFile nextChunkPutWithStyle: sourcesFile nextChunkText]. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
That fails as well. Help! What am I doing wrong? How does one copy the sources file properly and why do the above methods not work?
Cheers, - Andreas
On 2010-03-31, at 12:08 AM, Andreas Raab wrote:
That fails as well. Help! What am I doing wrong? How does one copy the sources file properly and why do the above methods not work?
Cheers,
- Andreas
Well both streams are those funky MultiByteFileStream
Soooo.
(a) your UTF8 data in the SourceFiles at: 1 is bad? (b) your MultiByteFileStream reader code is bad? (c) your MultiByteFileStream writer code is bad? (d) newFile position = sourcesFile size isn't what you think it is/does...
Or combinations of above.
Likely you want to run some nice text compare tool on the two files outside of squeak and see how sane things are.
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On Wed, 31 Mar 2010, Andreas Raab wrote:
Hi -
I was just trying to implement the code for condensing the changes to the end of the sources file and ran into some 'interesting' issues. I would expect this to work:
fullName := 'test1.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. FileDirectory default copyFile: sourcesFile toFile: newFile. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
This one is definitely a bug. Using a readOnlyCopy instead of the original sources file makes debugging a bit easier. :)
But it fails. Then I tried something else:
fullName := 'test2.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. [sourcesFile atEnd] whileFalse:[newFile nextChunkPut: sourcesFile nextChunk]. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
That fails to. Then I tried:
fullName := 'test3.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. [sourcesFile atEnd] whileFalse:[newFile nextChunkPutWithStyle: sourcesFile nextChunkText]. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
That fails as well. Help! What am I doing wrong? How does one copy the sources file properly and why do the above methods not work?
The latter two will probably fail, since the chunk reading code throws away unnecessary whitespaces.
I checked these in 4.0 and all three are failing the same way as in 4.1.
Levente
Cheers,
- Andreas
On Wed, 31 Mar 2010, Levente Uzonyi wrote:
On Wed, 31 Mar 2010, Andreas Raab wrote:
Hi -
I was just trying to implement the code for condensing the changes to the end of the sources file and ran into some 'interesting' issues. I would expect this to work:
fullName := 'test1.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. FileDirectory default copyFile: sourcesFile toFile: newFile. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
This one is definitely a bug. Using a readOnlyCopy instead of the original sources file makes debugging a bit easier. :)
Okay, found the cause of the issue. MultiByteFileStream doesn't implement it's own version of #nextInto: which is used by #copyFile:toFile:, therefore the file is not decoded. Replacing #nextInto: with an appropriate #next: send passes the test.
Implementing MultiByteFileStream >> #nextInto: is a bit tricky, since if the argument is a ByteString, then there's a chance that the result cannot be stored into the buffer. In similar cases the ByteString is swapped with a WideString, but in this case it can become a source of bugs, since the sender of #nextInto: may expect that the buffer is still a ByteString.
IMHO #copyFile:toFile: shouldn't exist, it's just a utility method, totally unrelated to FileDirectory. It also assumes that the files are in "ascii" mode which is wrong.
Levente
But it fails. Then I tried something else:
fullName := 'test2.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. [sourcesFile atEnd] whileFalse:[newFile nextChunkPut: sourcesFile nextChunk]. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
That fails to. Then I tried:
fullName := 'test3.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. [sourcesFile atEnd] whileFalse:[newFile nextChunkPutWithStyle: sourcesFile nextChunkText]. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
That fails as well. Help! What am I doing wrong? How does one copy the sources file properly and why do the above methods not work?
The latter two will probably fail, since the chunk reading code throws away unnecessary whitespaces.
I checked these in 4.0 and all three are failing the same way as in 4.1.
Levente
Cheers,
- Andreas
On 2010-03-31, at 3:59 PM, Levente Uzonyi wrote:
IMHO #copyFile:toFile: shouldn't exist, it's just a utility method, totally unrelated to FileDirectory. It also assumes that the files are in "ascii" mode which is wrong.
Levente
+1 In Sophie we replaced this with a primitive to which had the responsibility to ask the operating system to copy the files. The complexity of meta-data (security, dates/times, Access lists, stuff) on a *file* make Squeak's desire to copy just the bits rather 1970 ish... Oh wait pre that since it doesn't preserve the from file security settings either...
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On Wed, 31 Mar 2010, John M McIntosh wrote:
On 2010-03-31, at 3:59 PM, Levente Uzonyi wrote:
IMHO #copyFile:toFile: shouldn't exist, it's just a utility method, totally unrelated to FileDirectory. It also assumes that the files are in "ascii" mode which is wrong.
Levente
+1 In Sophie we replaced this with a primitive to which had the responsibility to ask the operating system to copy the files. The complexity of meta-data (security, dates/times, Access lists, stuff) on a *file* make Squeak's desire to copy just the bits rather 1970 ish... Oh wait pre that since it doesn't preserve the from file security settings either...
What's really "funny" about this method is that it doesn't really copy files, just the copies the contents of a ReadStream to WriteStream using a ByteString as a buffer. It doesn't use files at all.
Levente
--
John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On 2010-03-31, at 5:59 PM, Levente Uzonyi wrote:
+1 In Sophie we replaced this with a primitive to which had the responsibility to ask the operating system to copy the files. The complexity of meta-data (security, dates/times, Access lists, stuff) on a *file* make Squeak's desire to copy just the bits rather 1970 ish... Oh wait pre that since it doesn't preserve the from file security settings either...
What's really "funny" about this method is that it doesn't really copy files, just the copies the contents of a ReadStream to WriteStream using a ByteString as a buffer. It doesn't use files at all.
Levente
Ya, in Sophie everything was a URI. So in the copyFromURI:toURI we would check and say oh look both URI's are file based, so do this copyFile:to: primitive. Otherwise the URI read/write logic came into play...
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
The complexity of meta-data (security, dates/times, Access lists, stuff) on a *file* make Squeak's desire to copy just the bits rather 1970 ish...
Well, you could also say, it does it in a way that is platform-independent-ish... Lowest-common denominator, but the default unix cp command loses time-stamp and permissions as well unless you use -p...
Levente wrote:
What's really "funny" about this method is that it doesn't really copy files, just the copies the contents of a ReadStream to WriteStream using a ByteString as a buffer. It doesn't use files at all.
It is a "utility" method because it is called by API methods on FileDirectory. So perhaps #copyFile:toFile: should just be made private and then you can think of it as merely a composed-method.
Oh wait pre that since it doesn't preserve the from file security settings either...
--
John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On 3/31/2010 3:59 PM, Levente Uzonyi wrote:
Okay, found the cause of the issue. MultiByteFileStream doesn't implement it's own version of #nextInto: which is used by #copyFile:toFile:, therefore the file is not decoded. Replacing #nextInto: with an appropriate #next: send passes the test.
Thanks for the help, I've worked around the problem by using #next: and #nextPutAll: which seems to work. It's yucky though; if I hadn't added my sanity test I wouldn't even have found the problem and the sources file would've been silently broken.
Cheers, - Andreas
Implementing MultiByteFileStream >> #nextInto: is a bit tricky, since if the argument is a ByteString, then there's a chance that the result cannot be stored into the buffer. In similar cases the ByteString is swapped with a WideString, but in this case it can become a source of bugs, since the sender of #nextInto: may expect that the buffer is still a ByteString.
IMHO #copyFile:toFile: shouldn't exist, it's just a utility method, totally unrelated to FileDirectory. It also assumes that the files are in "ascii" mode which is wrong.
Levente
But it fails. Then I tried something else:
fullName := 'test2.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. [sourcesFile atEnd] whileFalse:[newFile nextChunkPut: sourcesFile nextChunk]. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
That fails to. Then I tried:
fullName := 'test3.sources'. "Copy sources file; change file name accordingly" FileStream forceNewFileNamed: fullName do:[:newFile| sourcesFile := SourceFiles at: 1. sourcesFile position: 0. [sourcesFile atEnd] whileFalse:[newFile nextChunkPutWithStyle: sourcesFile nextChunkText]. newFile position = sourcesFile size ifFalse:[self error: 'File copy failed']. ].
That fails as well. Help! What am I doing wrong? How does one copy the sources file properly and why do the above methods not work?
The latter two will probably fail, since the chunk reading code throws away unnecessary whitespaces.
I checked these in 4.0 and all three are failing the same way as in 4.1.
Levente
Cheers,
- Andreas
squeak-dev@lists.squeakfoundation.org
-
Andreas Raab -
Chris Muller -
John M McIntosh -
Levente Uzonyi