Hi Eliot.

Thank you, but I think I have my smalltalk solution while the latest bzcat was running.

"Concatenate two streams" took a bit of hacking and study....

its easy now that I see it..

I extended the OSPipe upToEnd with...

upToEnd: aString
"Answer the remaining elements in the string. This method is retained for backward
compatibility with older versions of CommandShell."

| strm s |
strm := WriteStream on: ''.
strm nextPutAll: aString.
[(s := self next: 2000) isEmpty
ifTrue: [^ strm contents]
ifFalse: [strm nextPutAll: s]] repeat


All I did is pass in the string I needed to prepend

|pipe |
Transcript clear.
pipe := UnixProcess bzcatAFileToPipe.
Transcript show:(pipe upToEnd:  '<?xml version="1.0" encoding="UTF-8"?>').

and it works.

<?xml version="1.0" encoding="UTF-8"?>
    <bookstore>

    <book category="COOKING">
      <title lang="en">Everyday Italian</title>
      <author>Giada De Laurentiis</author>
      <year>2005</year>
      <price>30.00</price>
    </book>

    <book category="CHILDREN">
      <title lang="en">Harry Potter</title>
      <author>J K. Rowling</author>
      <year>2005</year>
      <price>29.99</price>
    </book>

    <book category="WEB">
      <title lang="en">XQuery Kick Start</title>
      <author>James McGovern</author>
      <author>Per Bothner</author>
      <author>Kurt Cagle</author>
      <author>James Linn</author>
      <author>Vaidyanathan Nagarajan</author>
      <year>2003</year>
      <price>49.99</price>
    </book>

    <book category="WEB">
      <title lang="en">Learning XML</title>
      <author>Erik T. Ray</author>
      <year>2003</year>
      <price>39.95</price>
    </book>

    </bookstore>


I spent some time fighting

| strm s |
strm := WriteStream on: '<?xml version="1.0" encoding="UTF-8"?>'.
[(s := self next: 2000) isEmpty
ifTrue: [^ strm contents]
ifFalse: [strm nextPutAll: s]] repeat


but the <?xml version="1.0" encoding="UTF-8"?> kept getting erased from the strm.

no biggy, it works now, that's all that matters.

Also, I was returning one of the Reader or Writer from the OSPipe when I could have just returned the whole pipe...

next up is seeing how that works on the big file I have.

thx for the c- possibility should I need it.

cordially.







---- On Sat, 15 Jul 2023 14:00:32 -0400 Eliot Miranda <eliot.miranda@gmail.com> wrote ---

Hi Timothy,

    I think I’m right in thinking you don’t have enough disc space to copy the file.  So to get rid of the “dude” you’ll need to edit in place.

I would write a C program to do this.  I would test it on a small file first.  Here’s a very rough sketch (no error checking, rued direct on my phone)

#define BlockSize 1024
#define IndexOfDude 123 // needs to be the right value!!

int
main(int argc, char *argv[])
{
    int f = open(argv[1], O_RDWR);
  int n;
  char *buf = malloc(BlockSize);
  lseek(f, IndexOfDude + 4, SEEK_SET);
  while ((n = read(f, buf, BlockSize)) > 0) {
      lseek(f, 0 - (BlockSize + 4), SEEK_CUR);
      write(f, buf, BlockSize);
      lseek(f, 4, SEEK_CUR);
  }
  ftruncate(f, lseek(f, -4, SEEK_END);
  close(f);
  return 0;
}

If your file is > 2Gb you may need to use lseek64.  I hope the idea is clear.  You overwrite the bytes in the file ffom the start of “dude” with the bytes following “dude”, in chunks.  You could probably do this in Smalltalk also.  But the key is you only get one chance to get this right on the big file so test thoroughly on a smaller file and then ask what could possibly go wrong with integer sizes if your file is greater than 2^31-1 bytes big.
    
_,,,^..^,,,_ (phone)

On Jul 15, 2023, at 4:35 AM, gettimothy via Squeak-dev <squeak-dev@lists.squeakfoundation.org> wrote:


oops.


the bzcat method works for appending one bz2 to another.

Unfortunately, during the initial test runs, I concatted a bz2 file with the xml declaration to a bzip file with the word 'dude' in it.
Then, I used that file for the concatenation on the 'true' file.

<?xml version="1.0" encoding="UTF-8"?>
dude
<........rest of properlly formatted XML multi-gigabyte file here....



sigh






---- On Fri, 14 Jul 2023 11:40:42 -0400 gettimothy <gettimothy@zoho.com> wrote ---

Hi Folks.

I have a bzcat AttachableFileStream that I need to prepend some data to.

If I could concatenate two streams, it would look like this:

ios := ReadStream on:'<?xml version="1.0" encoding="UTF-8"?>
bzcat := "My AttachableFileStream".
fooios := ios, bzcat.

Nothing obvious showed up in my brief search.

anybody have any experience with this?

thx in advance.