Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The TraceDebugger [1] is a new back-in-time/time-travel/omniscient debugging tool for Squeak that allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacello new baseline: 'TraceDebugger'; repository: 'github://hpi-swa-lab/squeak-tracedebugger'; "repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';" "for Squeak 6.0" get; load.
*What can it do? (Features)*
- Record all method activations and historic states: Normally step through a program in the debugger while automatically recording its execution. - Replay execution of a traced program: Navigate through all method invocations using the *context tree* or the *Step Back/Step Forward* buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations). - Interact with historic states: Inspect/explore snapshots of objects or send them any message. - State-centric debugging using the *History Explorer*: Gather, explore, and visualize all changes to an object/expression over the recorded time ("When did this variable/collection/screenshot change?"). - Additional navigation tools for searching and filtering the context tree. - Focus on interactivity: No hours of recording, no GBs of mem consumption - at least for common small to medium programs. - UI resembles the classic Smalltalk debugger: You'll find your familiar stepping buttons, code browsing tools, inspectors, and shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully used it to understand several bugs and interaction patterns in the Trunk (Morphic layout/rendering, compiler/decompiler, code simulation, …). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
*What can‘t it do (yet)? (Limitations and future work)*
- High performance: While (sufficiently) fast enough for most small to medium workloads, tracing very compute- or mem-intensive operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m, complex rendering: minutes up to hours). - Not a dataflow analyzer: The TraceDebugger does not track dataflow events (e.g., argument passing) but only state changes. - No tracing of external states/events for FFI/OSProcess or custom VM modules. - No support for advanced language concepts such as identity forwarding/write barriers.
*How does it work? (Implementation)*
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing extensions by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized code simulator that overrides instructions for sending messages (e.g., send, superSend) and for performing side-effects (e.g., popIntoRcvr, primitiveAtPut, push). All message sends are recorded in a tree and all changed object slots are stored in a sparse time-dependent memory structure before they are overwritten. For time-traveling, the tree is traversed using a cursor. For accessing historic objects, a proxy evaluates all messages sent to an object in another specialized simulator (retracing simulator) that emulates historic states for the requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering state changes in the History Explorer efficiently, the query is evaluated in a range retracing simulator with vectorization and fork semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its implementation and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger and TDBCursor should provide good starting points.
*How can I use it?*
Please try it out and report feedback! The TraceDebugger supports the latest Squeak Trunk and Squeak 6.0. You can either download a prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacello new baseline: 'TraceDebugger'; repository: 'github://hpi-swa-lab/squeak-tracedebugger'; "repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';" "for Squeak 6.0" get; load.
To get started, just open a normal debugger (e.g., by selecting an expression and pressing Cmd+Shift+D to debug it) and then press the "Trace It" button on the right. There‘s also some pretty detailed documentation in the Help Browser that covers everything you should know.
My goal is to improve convenience and provide a useful tool for the community, so I‘m very excited to hear your impressions, ideas, and thoughts. Here, on GitHub, or in a private message. Let‘s have a great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of "subclassing from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Object-Centric Time-Travel Debugging: Exploring Traces of Objects. In Companion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming (<Programming>'23 Companion), March 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages. DOI: 10.1145/3594671.3594678. PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging. In Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! '23), October 25–27, 2023, Cascais, Portugal. ACM, New York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5] http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.h...
--- Sent from Squeak Inbox Talk
This sounds like really interesting work! I love the idea of being able to interactively go back in "oops, I‘ve stepped too far, let‘s start all over again" situations. It will probably take some time for me and others to wrap our heads around the things you have done, so don't be surprised if you get a delayed response to this announcement :-)
Congratulations! Dave
On Fri, Dec 29 2023 at 01:42:16 AM +0100, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The *TraceDebugger* [1] is a new back-in-time/time-travel/omniscient debugging tool for Squeak that allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for Squeak 6.0"/ get; load.
**What can it do? (Features)**
- *Record all method activations and historic states:* Normally step
through a program in the debugger while automatically recording its execution.
- *Replay execution of a traced program:* Navigate through all
method invocations using the /*context tree*/ or the /*Step Back/Step Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations).
- *Interact with historic states:* Inspect/explore snapshots of
objects or send them any message.
- *State-centric debugging using the ***/History Explorer*/**:*
Gather, explore, and visualize all changes to an object/expression over the recorded time ("When did this variable/collection/screenshot change?").
- *Additional navigation tools* for searching and filtering the
context tree.
- *Focus on interactivity:* No hours of recording, no GBs of mem
consumption - at least for common small to medium programs.
- *UI resembles the classic Smalltalk debugger:* You'll find your
familiar stepping buttons, code browsing tools, inspectors, and shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully used it to understand several bugs and interaction patterns in the Trunk (Morphic layout/rendering, compiler/decompiler, code simulation, …). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
**What can‘t it do (yet)? (Limitations and future work)**
- *High performance:* While (sufficiently) fast enough for most
small to medium workloads, tracing very compute- or mem-intensive operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m, complex rendering: minutes up to hours).
- *Not a dataflow analyzer:* The TraceDebugger does not track
dataflow events (e.g., argument passing) but only state changes.
- *No tracing of external states/events* for FFI/OSProcess or custom
VM modules.
- *No support for advanced language concepts* such as identity
forwarding/write barriers.
**How does it work? (Implementation)**
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing extensions by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized code simulator that overrides instructions for sending messages (e.g., send, superSend) and for performing side-effects (e.g., popIntoRcvr, primitiveAtPut, push). All message sends are recorded in a tree and all changed object slots are stored in a sparse time-dependent memory structure before they are overwritten. For time-traveling, the tree is traversed using a cursor. For accessing historic objects, a proxy evaluates all messages sent to an object in another specialized simulator (retracing simulator) that emulates historic states for the requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering state changes in the History Explorer efficiently, the query is evaluated in a range retracing simulator with vectorization and fork semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its implementation and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger code://TraceDebugger and TDBCursor code://TDBCursor should provide good starting points.
**How can I use it?**
Please try it out and report feedback! The TraceDebugger supports the latest Squeak Trunk and Squeak 6.0. You can either download a prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for Squeak 6.0"/ get; load.
To get started, just open a normal debugger (e.g., by selecting an expression and pressing Cmd+Shift+D to debug it) and then press the "Trace It" button on the right. There‘s also some pretty detailed documentation in the Help Browser <code:// TraceDebugger showHelp> that covers everything you should know.
My goal is to improve convenience and provide a useful tool for the community, so I‘m very excited to hear your impressions, ideas, and thoughts. Here, on GitHub, or in a private message. Let‘s have a great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of "subclassing from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Object-Centric Time-Travel Debugging: Exploring Traces of Objects. https://doi.org/10.1145/3594671.3594678 In /Companion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming/ (/<Programming>'23 Companion/), March 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages. DOI: 10.1145/3594671.3594678 https://doi.org/10.1145/3594671.3594678. PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In /Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software/ (/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM, New York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 https://doi.org/10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5] http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
/Sent from//Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk/
Thanks for the reply, Dave! I will try to post one or two concrete use cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2023-12-29T11:01:10-06:00, lewis@mail.msen.com wrote:
This sounds like really interesting work! I love the idea of being able to interactively go back in "oops, I‘ve stepped too far, let‘s start all over again" situations. It will probably take some time for me and others to wrap our heads around the things you have done, so don't be surprised if you get a delayed response to this announcement :-)
Congratulations! Dave
On Fri, Dec 29 2023 at 01:42:16 AM +0100, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The *TraceDebugger* [1] is a new back-in-time/time-travel/omniscient debugging tool for Squeak that allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for Squeak 6.0"/ get; load.
**What can it do? (Features)**
- *Record all method activations and historic states:* Normally step
through a program in the debugger while automatically recording its execution.
- *Replay execution of a traced program:* Navigate through all
method invocations using the /*context tree*/ or the /*Step Back/Step Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations).
- *Interact with historic states:* Inspect/explore snapshots of
objects or send them any message.
- *State-centric debugging using the ***/History Explorer*/**:*
Gather, explore, and visualize all changes to an object/expression over the recorded time ("When did this variable/collection/screenshot change?").
- *Additional navigation tools* for searching and filtering the
context tree.
- *Focus on interactivity:* No hours of recording, no GBs of mem
consumption - at least for common small to medium programs.
- *UI resembles the classic Smalltalk debugger:* You'll find your
familiar stepping buttons, code browsing tools, inspectors, and shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully used it to understand several bugs and interaction patterns in the Trunk (Morphic layout/rendering, compiler/decompiler, code simulation, …). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
**What can‘t it do (yet)? (Limitations and future work)**
- *High performance:* While (sufficiently) fast enough for most
small to medium workloads, tracing very compute- or mem-intensive operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m, complex rendering: minutes up to hours).
- *Not a dataflow analyzer:* The TraceDebugger does not track
dataflow events (e.g., argument passing) but only state changes.
- *No tracing of external states/events* for FFI/OSProcess or custom
VM modules.
- *No support for advanced language concepts* such as identity
forwarding/write barriers.
**How does it work? (Implementation)**
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing extensions by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized code simulator that overrides instructions for sending messages (e.g., send, superSend) and for performing side-effects (e.g., popIntoRcvr, primitiveAtPut, push). All message sends are recorded in a tree and all changed object slots are stored in a sparse time-dependent memory structure before they are overwritten. For time-traveling, the tree is traversed using a cursor. For accessing historic objects, a proxy evaluates all messages sent to an object in another specialized simulator (retracing simulator) that emulates historic states for the requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering state changes in the History Explorer efficiently, the query is evaluated in a range retracing simulator with vectorization and fork semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its implementation and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger code://TraceDebugger and TDBCursor code://TDBCursor should provide good starting points.
**How can I use it?**
Please try it out and report feedback! The TraceDebugger supports the latest Squeak Trunk and Squeak 6.0. You can either download a prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for Squeak 6.0"/ get; load.
To get started, just open a normal debugger (e.g., by selecting an expression and pressing Cmd+Shift+D to debug it) and then press the "Trace It" button on the right. There‘s also some pretty detailed documentation in the Help Browser <code:// TraceDebugger showHelp> that covers everything you should know.
My goal is to improve convenience and provide a useful tool for the community, so I‘m very excited to hear your impressions, ideas, and thoughts. Here, on GitHub, or in a private message. Let‘s have a great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of "subclassing from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Object-Centric Time-Travel Debugging: Exploring Traces of Objects. https://doi.org/10.1145/3594671.3594678 In /Companion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming/ (/<Programming>'23 Companion/), March 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages. DOI: 10.1145/3594671.3594678 https://doi.org/10.1145/3594671.3594678. PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In /Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software/ (/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM, New York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 https://doi.org/10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5] http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
/Sent from//Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing your use cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been working with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it stops at Context>>terminate and the view gets corrupted (the initial part of the trace is hidden and can't be made visible unless clicking on some of the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of error:
Maybe this is just an unfortunate example... Or maybe I'm just doing something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Thanks for the reply, Dave! I will try to post one or two concrete use cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis@mail.msen.com wrote:
This sounds like really interesting work! I love the idea of being able to interactively go back in "oops, I‘ve stepped too far, let‘s start all over again" situations. It will probably take some time for me and others to wrap our heads around the things you have done, so don't be surprised if you get a delayed response to this announcement :-)
Congratulations! Dave
On Fri, Dec 29 2023 at 01:42:16 AM +0100, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The *TraceDebugger* [1] is a
new
back-in-time/time-travel/omniscient debugging tool for Squeak that allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for Squeak 6.0"/ get; load.
**What can it do? (Features)**
- *Record all method activations and historic states:* Normally
step
through a program in the debugger while automatically recording its execution.
- *Replay execution of a traced program:* Navigate through all
method invocations using the /*context tree*/ or the /*Step
Back/Step
Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations).
- *Interact with historic states:* Inspect/explore snapshots of
objects or send them any message.
- *State-centric debugging using the ***/History Explorer*/**:*
Gather, explore, and visualize all changes to an object/expression over the recorded time ("When did this
variable/collection/screenshot
change?").
- *Additional navigation tools* for searching and filtering the
context tree.
- *Focus on interactivity:* No hours of recording, no GBs of mem
consumption - at least for common small to medium programs.
- *UI resembles the classic Smalltalk debugger:* You'll find your
familiar stepping buttons, code browsing tools, inspectors, and shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully used
it
to understand several bugs and interaction patterns in the Trunk (Morphic layout/rendering, compiler/decompiler, code simulation, …). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
**What can‘t it do (yet)? (Limitations and future work)**
- *High performance:* While (sufficiently) fast enough for most
small to medium workloads, tracing very compute- or mem-intensive operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m, complex rendering: minutes up to hours).
- *Not a dataflow analyzer:* The TraceDebugger does not track
dataflow events (e.g., argument passing) but only state changes.
- *No tracing of external states/events* for FFI/OSProcess or
custom
VM modules.
- *No support for advanced language concepts* such as identity
forwarding/write barriers.
**How does it work? (Implementation)**
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing extensions by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized code simulator that overrides instructions for sending messages (e.g., send, superSend) and for performing side-effects (e.g.,
popIntoRcvr,
primitiveAtPut, push). All message sends are recorded in a tree and all changed object slots are stored in a sparse time-dependent
memory
structure before they are overwritten. For time-traveling, the tree is traversed using a cursor. For accessing historic objects, a
proxy
evaluates all messages sent to an object in another specialized simulator (retracing simulator) that emulates historic states for
the
requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering state changes in the History Explorer efficiently, the query is evaluated in a range retracing simulator with vectorization and fork
semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its implementation and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger
and TDBCursor code://TDBCursor should provide good starting
points.
**How can I use it?**
Please try it out and report feedback! The TraceDebugger supports the latest Squeak Trunk and Squeak 6.0. You can either download a prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for Squeak 6.0"/ get; load.
To get started, just open a normal debugger (e.g., by selecting an expression and pressing Cmd+Shift+D to debug it) and then press the "Trace It" button on the right. There‘s also some pretty detailed documentation in the Help Browser <code:// TraceDebugger showHelp> that covers everything you should know.
My goal is to improve convenience and provide a useful tool for the community, so I‘m very excited to hear your impressions, ideas, and thoughts. Here, on GitHub, or in a private message. Let‘s have a great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of "subclassing from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Object-Centric Time-Travel Debugging: Exploring Traces of Objects. https://doi.org/10.1145/3594671.3594678 In /Companion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming/ (/<Programming>'23 Companion/), March 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages. DOI: 10.1145/3594671.3594678 https://doi.org/10.1145/3594671.3594678. PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Time-Awareness in Object Exploration Tools: Toward In Situ
Omniscient
Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In /Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software/ (/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM, New York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 https://doi.org/10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5]
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
/Sent from//Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me. :-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is not ready for all your clever investigations regarding non-local returns and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards programs with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single global tree - its structure would change over the execution time, and when selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack at the viewed point in time (see also the '@ <timeIndex>' in the window title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You would notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to step out of #terminateTo: again. (You can update the TraceDebugger from the window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step further and eventually return back to the starting point. :-) If you want to, you can also turn off the preference "Show call tree in TraceDebugger" to make the TraceDebugger look more like a normal debugger, which also solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it removes one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the TraceDebugger. Better use it to explore how the simulator works. :-) For example, you could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method, press Cmd + f(ind), and type "return:from:" to investigate the behavior of your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort some things out! :-)
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2023-12-30T19:04:07+00:00, mail@jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing your use cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been working with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it stops at Context>>terminate and the view gets corrupted (the initial part of the trace is hidden and can't be made visible unless clicking on some of the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of error:
Maybe this is just an unfortunate example... Or maybe I'm just doing something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Thanks for the reply, Dave! I will try to post one or two concrete use cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote:
This sounds like really interesting work! I love the idea of being able to interactively go back in "oops, I‘ve stepped too far, let‘s start all over again" situations. It will probably take some time for me and others to wrap our heads around the things you have done, so don't be surprised if you get a delayed response to this announcement :-)
Congratulations! Dave
On Fri, Dec 29 2023 at 01:42:16 AM +0100, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The *TraceDebugger* [1] is a
new
back-in-time/time-travel/omniscient debugging tool for Squeak that allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for Squeak 6.0"/ get; load.
**What can it do? (Features)**
- *Record all method activations and historic states:* Normally
step
through a program in the debugger while automatically recording its execution.
- *Replay execution of a traced program:* Navigate through all
method invocations using the /*context tree*/ or the /*Step
Back/Step
Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations).
- *Interact with historic states:* Inspect/explore snapshots of
objects or send them any message.
- *State-centric debugging using the ***/History Explorer*/**:*
Gather, explore, and visualize all changes to an object/expression over the recorded time ("When did this
variable/collection/screenshot
change?").
- *Additional navigation tools* for searching and filtering the
context tree.
- *Focus on interactivity:* No hours of recording, no GBs of mem
consumption - at least for common small to medium programs.
- *UI resembles the classic Smalltalk debugger:* You'll find your
familiar stepping buttons, code browsing tools, inspectors, and shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully used
it
to understand several bugs and interaction patterns in the Trunk (Morphic layout/rendering, compiler/decompiler, code simulation, …). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
**What can‘t it do (yet)? (Limitations and future work)**
- *High performance:* While (sufficiently) fast enough for most
small to medium workloads, tracing very compute- or mem-intensive operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m, complex rendering: minutes up to hours).
- *Not a dataflow analyzer:* The TraceDebugger does not track
dataflow events (e.g., argument passing) but only state changes.
- *No tracing of external states/events* for FFI/OSProcess or
custom
VM modules.
- *No support for advanced language concepts* such as identity
forwarding/write barriers.
**How does it work? (Implementation)**
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing extensions by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized code simulator that overrides instructions for sending messages (e.g., send, superSend) and for performing side-effects (e.g.,
popIntoRcvr,
primitiveAtPut, push). All message sends are recorded in a tree and all changed object slots are stored in a sparse time-dependent
memory
structure before they are overwritten. For time-traveling, the tree is traversed using a cursor. For accessing historic objects, a
proxy
evaluates all messages sent to an object in another specialized simulator (retracing simulator) that emulates historic states for
the
requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering state changes in the History Explorer efficiently, the query is evaluated in a range retracing simulator with vectorization and fork
semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its implementation and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger
and TDBCursor code://TDBCursor should provide good starting
points.
**How can I use it?**
Please try it out and report feedback! The TraceDebugger supports the latest Squeak Trunk and Squeak 6.0. You can either download a prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for Squeak 6.0"/ get; load.
To get started, just open a normal debugger (e.g., by selecting an expression and pressing Cmd+Shift+D to debug it) and then press the "Trace It" button on the right. There‘s also some pretty detailed documentation in the Help Browser <code:// TraceDebugger showHelp> that covers everything you should know.
My goal is to improve convenience and provide a useful tool for the community, so I‘m very excited to hear your impressions, ideas, and thoughts. Here, on GitHub, or in a private message. Let‘s have a great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of "subclassing from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Object-Centric Time-Travel Debugging: Exploring Traces of Objects. https://doi.org/10.1145/3594671.3594678 In /Companion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming/ (/<Programming>'23 Companion/), March 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages. DOI: 10.1145/3594671.3594678 https://doi.org/10.1145/3594671.3594678. PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Time-Awareness in Object Exploration Tools: Toward In Situ
Omniscient
Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In /Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software/ (/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM, New York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 https://doi.org/10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5]
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
/Sent from//Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Christoph,
a small issue:
IndentingListItemMorph>>openPath:
contains a line:
ifNil: [(each complexContents asString sameAs: anArray first)
where
each complexContents asString
returns text but sameAs: is not defined for Text, and the line then causes an MNU.
I'd suggest adding
Text>>sameAs: aString ^string sameAs: aString
The MNU error happens when tracing
[^2] ensure: []
and just stepping over until the error occurs.
Best, Jaromir
On 31-Dec-23 2:16:32 AM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me. :-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is not ready for all your clever investigations regarding non-local returns and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards programs with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single global tree
- its structure would change over the execution time, and when
selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack at the viewed point in time (see also the '@ <timeIndex>' in the window title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You would notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to step out of #terminateTo: again. (You can update the TraceDebugger from the window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step further and eventually return back to the starting point. :-) If you want to, you can also turn off the preference "Show call tree in TraceDebugger" to make the TraceDebugger look more like a normal debugger, which also solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it removes one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the TraceDebugger. Better use it to explore how the simulator works. :-) For example, you could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method, press Cmd + f(ind), and type "return:from:" to investigate the behavior of your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort some things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail@jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it
stops
at Context>>terminate and the view gets corrupted (the initial part
of
the trace is hidden and can't be made visible unless clicking on some
of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of error:
Maybe this is just an unfortunate example... Or maybe I'm just doing something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Thanks for the reply, Dave! I will try to post one or two concrete
use
cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote:
This sounds like really interesting work! I love the idea of
being
able to interactively go back in "oops, I‘ve stepped too far,
let‘s
start all over again" situations. It will probably take some time
for
me and others to wrap our heads around the things you have done,
so
don't be surprised if you get a delayed response to this
announcement
:-)
Congratulations! Dave
On Fri, Dec 29 2023 at 01:42:16 AM +0100, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The *TraceDebugger* [1] is
a
new
back-in-time/time-travel/omniscient debugging tool for Squeak
that
allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
6.0"/ get; load.
**What can it do? (Features)**
- *Record all method activations and historic states:* Normally
step
through a program in the debugger while automatically recording
its
execution.
- *Replay execution of a traced program:* Navigate through all
method invocations using the /*context tree*/ or the /*Step
Back/Step
Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations).
- *Interact with historic states:* Inspect/explore snapshots of
objects or send them any message.
- *State-centric debugging using the ***/History Explorer*/**:*
Gather, explore, and visualize all changes to an
object/expression
over the recorded time ("When did this
variable/collection/screenshot
change?").
- *Additional navigation tools* for searching and filtering the
context tree.
- *Focus on interactivity:* No hours of recording, no GBs of
mem
consumption - at least for common small to medium programs.
- *UI resembles the classic Smalltalk debugger:* You'll find
your
familiar stepping buttons, code browsing tools, inspectors, and shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully
used
it
to understand several bugs and interaction patterns in the
Trunk
(Morphic layout/rendering, compiler/decompiler, code
simulation,
…). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
**What can‘t it do (yet)? (Limitations and future work)**
- *High performance:* While (sufficiently) fast enough for most
small to medium workloads, tracing very compute- or
mem-intensive
operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m,
complex
rendering: minutes up to hours).
- *Not a dataflow analyzer:* The TraceDebugger does not track
dataflow events (e.g., argument passing) but only state
changes.
- *No tracing of external states/events* for FFI/OSProcess or
custom
VM modules.
- *No support for advanced language concepts* such as identity
forwarding/write barriers.
**How does it work? (Implementation)**
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing
extensions
by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized code simulator that overrides instructions for sending messages
(e.g.,
send, superSend) and for performing side-effects (e.g.,
popIntoRcvr,
primitiveAtPut, push). All message sends are recorded in a tree
and
all changed object slots are stored in a sparse time-dependent
memory
structure before they are overwritten. For time-traveling, the
tree
is traversed using a cursor. For accessing historic objects, a
proxy
evaluates all messages sent to an object in another specialized simulator (retracing simulator) that emulates historic states
for
the
requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering
state
changes in the History Explorer efficiently, the query is
evaluated
in a range retracing simulator with vectorization and fork
semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its
implementation
and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger
and TDBCursor code://TDBCursor should provide good starting
points.
**How can I use it?**
Please try it out and report feedback! The TraceDebugger
supports
the latest Squeak Trunk and Squeak 6.0. You can either download
a
prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
6.0"/ get; load.
To get started, just open a normal debugger (e.g., by selecting
an
expression and pressing Cmd+Shift+D to debug it) and then press
the
"Trace It" button on the right. There‘s also some pretty
detailed
documentation in the Help Browser <code:// TraceDebugger
showHelp>
that covers everything you should know.
My goal is to improve convenience and provide a useful tool for
the
community, so I‘m very excited to hear your impressions, ideas,
and
thoughts. Here, on GitHub, or in a private message. Let‘s have
a
great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of
"subclassing
from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
Object-Centric Time-Travel Debugging: Exploring Traces of
Objects.
https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
of the 7th International Conference on the Art, Science, and Engineering of Programming/ (/<Programming>'23 Companion/),
March
13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages.
DOI:
10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
Time-Awareness in Object Exploration Tools: Toward In Situ
Omniscient
Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In /Proceedings of the 2023 ACM SIGPLAN International Symposium on
New
Ideas, New Paradigms, and Reflections on Programming and
Software/
(/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM,
New
York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 https://doi.org/10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5]
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
/Sent from//Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would be the best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would help tremendously to be able to ask right away instead of trudging through the code/help :) The code usually helps to understand **how** things work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in TraceDebugger"
Thanks, that helps to familiarize myself with the new functionalities "step-by-step", and not be overwhelmed by all the might of the call tree :) Being able to go back is already a hell of an improvement! Or when you open an inspector on a context the context's state gets frozen in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the primitive call. The Trace debugger, however, starts executing the fallback code of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me. :-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is not ready for all your clever investigations regarding non-local returns and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards programs with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single global tree
- its structure would change over the execution time, and when
selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack at the viewed point in time (see also the '@ <timeIndex>' in the window title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You would notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to step out of #terminateTo: again. (You can update the TraceDebugger from the window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step further and eventually return back to the starting point. :-) If you want to, you can also turn off the preference "Show call tree in TraceDebugger" to make the TraceDebugger look more like a normal debugger, which also solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it removes one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the TraceDebugger. Better use it to explore how the simulator works. :-) For example, you could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method, press Cmd + f(ind), and type "return:from:" to investigate the behavior of your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort some things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail@jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it
stops
at Context>>terminate and the view gets corrupted (the initial part
of
the trace is hidden and can't be made visible unless clicking on some
of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of error:
Maybe this is just an unfortunate example... Or maybe I'm just doing something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Thanks for the reply, Dave! I will try to post one or two concrete
use
cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote:
This sounds like really interesting work! I love the idea of
being
able to interactively go back in "oops, I‘ve stepped too far,
let‘s
start all over again" situations. It will probably take some time
for
me and others to wrap our heads around the things you have done,
so
don't be surprised if you get a delayed response to this
announcement
:-)
Congratulations! Dave
On Fri, Dec 29 2023 at 01:42:16 AM +0100, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The *TraceDebugger* [1] is
a
new
back-in-time/time-travel/omniscient debugging tool for Squeak
that
allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
6.0"/ get; load.
**What can it do? (Features)**
- *Record all method activations and historic states:* Normally
step
through a program in the debugger while automatically recording
its
execution.
- *Replay execution of a traced program:* Navigate through all
method invocations using the /*context tree*/ or the /*Step
Back/Step
Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations).
- *Interact with historic states:* Inspect/explore snapshots of
objects or send them any message.
- *State-centric debugging using the ***/History Explorer*/**:*
Gather, explore, and visualize all changes to an
object/expression
over the recorded time ("When did this
variable/collection/screenshot
change?").
- *Additional navigation tools* for searching and filtering the
context tree.
- *Focus on interactivity:* No hours of recording, no GBs of
mem
consumption - at least for common small to medium programs.
- *UI resembles the classic Smalltalk debugger:* You'll find
your
familiar stepping buttons, code browsing tools, inspectors, and shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully
used
it
to understand several bugs and interaction patterns in the
Trunk
(Morphic layout/rendering, compiler/decompiler, code
simulation,
…). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
**What can‘t it do (yet)? (Limitations and future work)**
- *High performance:* While (sufficiently) fast enough for most
small to medium workloads, tracing very compute- or
mem-intensive
operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m,
complex
rendering: minutes up to hours).
- *Not a dataflow analyzer:* The TraceDebugger does not track
dataflow events (e.g., argument passing) but only state
changes.
- *No tracing of external states/events* for FFI/OSProcess or
custom
VM modules.
- *No support for advanced language concepts* such as identity
forwarding/write barriers.
**How does it work? (Implementation)**
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing
extensions
by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized code simulator that overrides instructions for sending messages
(e.g.,
send, superSend) and for performing side-effects (e.g.,
popIntoRcvr,
primitiveAtPut, push). All message sends are recorded in a tree
and
all changed object slots are stored in a sparse time-dependent
memory
structure before they are overwritten. For time-traveling, the
tree
is traversed using a cursor. For accessing historic objects, a
proxy
evaluates all messages sent to an object in another specialized simulator (retracing simulator) that emulates historic states
for
the
requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering
state
changes in the History Explorer efficiently, the query is
evaluated
in a range retracing simulator with vectorization and fork
semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its
implementation
and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger
and TDBCursor code://TDBCursor should provide good starting
points.
**How can I use it?**
Please try it out and report feedback! The TraceDebugger
supports
the latest Squeak Trunk and Squeak 6.0. You can either download
a
prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
6.0"/ get; load.
To get started, just open a normal debugger (e.g., by selecting
an
expression and pressing Cmd+Shift+D to debug it) and then press
the
"Trace It" button on the right. There‘s also some pretty
detailed
documentation in the Help Browser <code:// TraceDebugger
showHelp>
that covers everything you should know.
My goal is to improve convenience and provide a useful tool for
the
community, so I‘m very excited to hear your impressions, ideas,
and
thoughts. Here, on GitHub, or in a private message. Let‘s have
a
great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of
"subclassing
from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
Object-Centric Time-Travel Debugging: Exploring Traces of
Objects.
https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
of the 7th International Conference on the Art, Science, and Engineering of Programming/ (/<Programming>'23 Companion/),
March
13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages.
DOI:
10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
Time-Awareness in Object Exploration Tools: Toward In Situ
Omniscient
Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In /Proceedings of the 2023 ACM SIGPLAN International Symposium on
New
Ideas, New Paradigms, and Reflections on Programming and
Software/
(/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM,
New
York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 https://doi.org/10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5]
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
/Sent from//Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to the traditional one?
In other words: if I run the traditional debugger and the Trace one side by side, should they display analogous steps?
In the other message I wrote about a different way to simulate primitive calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" mail@jaromir.net wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would be the best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would help tremendously to be able to ask right away instead of trudging through the code/help :) The code usually helps to understand **how** things work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new functionalities "step-by-step", and not be overwhelmed by all the might of the call tree :) Being able to go back is already a hell of an improvement! Or when you open an inspector on a context the context's state gets frozen in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the primitive call. The Trace debugger, however, starts executing the fallback code of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me. :-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is not ready for all your clever investigations regarding non-local returns and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards programs with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single global tree - its structure would change over the execution time, and when selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack at the viewed point in time (see also the '@ <timeIndex>' in the window title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You would notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to step out of #terminateTo: again. (You can update the TraceDebugger from the window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step further and eventually return back to the starting point. :-) If you want to, you can also turn off the preference "Show call tree in TraceDebugger" to make the TraceDebugger look more like a normal debugger, which also solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it removes one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the TraceDebugger. Better use it to explore how the simulator works. :-) For example, you could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method, press Cmd + f(ind), and type "return:from:" to investigate the behavior of your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort some things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail@jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it
stops
at Context>>terminate and the view gets corrupted (the initial part
of
the trace is hidden and can't be made visible unless clicking on
some of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of error:
Maybe this is just an unfortunate example... Or maybe I'm just doing something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Thanks for the reply, Dave! I will try to post one or two concrete
use
cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote:
This sounds like really interesting work! I love the idea of
being
able to interactively go back in "oops, I‘ve stepped too far,
let‘s
start all over again" situations. It will probably take some
time for
me and others to wrap our heads around the things you have done,
so
don't be surprised if you get a delayed response to this
announcement
:-)
Congratulations! Dave
On Fri, Dec 29 2023 at 01:42:16 AM +0100, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The *TraceDebugger* [1] is
a
new
back-in-time/time-travel/omniscient debugging tool for Squeak
that
allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
6.0"/ get; load.
**What can it do? (Features)**
- *Record all method activations and historic states:*
Normally
step
through a program in the debugger while automatically
recording its
execution.
- *Replay execution of a traced program:* Navigate through all
method invocations using the /*context tree*/ or the /*Step
Back/Step
Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations).
- *Interact with historic states:* Inspect/explore snapshots
of
objects or send them any message.
- *State-centric debugging using the ***/History
Explorer*/**:*
Gather, explore, and visualize all changes to an
object/expression
over the recorded time ("When did this
variable/collection/screenshot
change?").
- *Additional navigation tools* for searching and filtering
the
context tree.
- *Focus on interactivity:* No hours of recording, no GBs of
mem
consumption - at least for common small to medium programs.
- *UI resembles the classic Smalltalk debugger:* You'll find
your
familiar stepping buttons, code browsing tools, inspectors,
and
shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully
used
it
to understand several bugs and interaction patterns in the
Trunk
(Morphic layout/rendering, compiler/decompiler, code
simulation,
…). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
**What can‘t it do (yet)? (Limitations and future work)**
- *High performance:* While (sufficiently) fast enough for
most
small to medium workloads, tracing very compute- or
mem-intensive
operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m,
complex
rendering: minutes up to hours).
- *Not a dataflow analyzer:* The TraceDebugger does not track
dataflow events (e.g., argument passing) but only state
changes.
- *No tracing of external states/events* for FFI/OSProcess or
custom
VM modules.
- *No support for advanced language concepts* such as identity
forwarding/write barriers.
**How does it work? (Implementation)**
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing
extensions
by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized
code
simulator that overrides instructions for sending messages
(e.g.,
send, superSend) and for performing side-effects (e.g.,
popIntoRcvr,
primitiveAtPut, push). All message sends are recorded in a
tree and
all changed object slots are stored in a sparse time-dependent
memory
structure before they are overwritten. For time-traveling, the
tree
is traversed using a cursor. For accessing historic objects, a
proxy
evaluates all messages sent to an object in another
specialized
simulator (retracing simulator) that emulates historic states
for
the
requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering
state
changes in the History Explorer efficiently, the query is
evaluated
in a range retracing simulator with vectorization and fork
semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its
implementation
and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger
and TDBCursor code://TDBCursor should provide good starting
points.
**How can I use it?**
Please try it out and report feedback! The TraceDebugger
supports
the latest Squeak Trunk and Squeak 6.0. You can either
download a
prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacellonew baseline:'TraceDebugger'; repository:'github://hpi-swa-lab/squeak-tracedebugger'; /"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
6.0"/ get; load.
To get started, just open a normal debugger (e.g., by
selecting an
expression and pressing Cmd+Shift+D to debug it) and then
press the
"Trace It" button on the right. There‘s also some pretty
detailed
documentation in the Help Browser <code:// TraceDebugger
showHelp>
that covers everything you should know.
My goal is to improve convenience and provide a useful tool
for the
community, so I‘m very excited to hear your impressions,
ideas, and
thoughts. Here, on GitHub, or in a private message. Let‘s have
a
great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of
"subclassing
from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
Object-Centric Time-Travel Debugging: Exploring Traces of
Objects.
https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
of the 7th International Conference on the Art, Science, and Engineering of Programming/ (/<Programming>'23 Companion/),
March
13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages.
DOI:
10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
Time-Awareness in Object Exploration Tools: Toward In Situ
Omniscient
Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In /Proceedings of the 2023 ACM SIGPLAN International Symposium
on New
Ideas, New Paradigms, and Reflections on Programming and
Software/
(/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM,
New
York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 https://doi.org/10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5]
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
/Sent from//Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Jaromir,
thanks again for your messages, please keep them coming! And squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me. But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: Context runSimulated: [Context runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate Simulator debug: [thisContext findNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContext findNextHandlerContextStarting] debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContext class] debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulator debug: [thisContext findNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Example: do step through to the [^2] block and then step through again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail@jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to the traditional one?
In other words: if I run the traditional debugger and the Trace one side by side, should they display analogous steps?
In the other message I wrote about a different way to simulate primitive calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net> wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would be the best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would help tremendously to be able to ask right away instead of trudging through the code/help :) The code usually helps to understand **how** things work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new functionalities "step-by-step", and not be overwhelmed by all the might of the call tree :) Being able to go back is already a hell of an improvement! Or when you open an inspector on a context the context's state gets frozen in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the primitive call. The Trace debugger, however, starts executing the fallback code of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me. :-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is not ready for all your clever investigations regarding non-local returns and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards programs with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single global tree - its structure would change over the execution time, and when selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack at the viewed point in time (see also the '@ <timeIndex>' in the window title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You would notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to step out of #terminateTo: again. (You can update the TraceDebugger from the window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step further and eventually return back to the starting point. :-) If you want to, you can also turn off the preference "Show call tree in TraceDebugger" to make the TraceDebugger look more like a normal debugger, which also solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it removes one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the TraceDebugger. Better use it to explore how the simulator works. :-) For example, you could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method, press Cmd + f(ind), and type "return:from:" to investigate the behavior of your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort some things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it
stops
at Context>>terminate and the view gets corrupted (the initial part
of
the trace is hidden and can't be made visible unless clicking on
some of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of error:
Maybe this is just an unfortunate example... Or maybe I'm just doing something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Thanks for the reply, Dave! I will try to post one or two concrete
use
cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote:
This sounds like really interesting work! I love the idea of
being
able to interactively go back in "oops, I‘ve stepped too far,
let‘s
start all over again" situations. It will probably take some
time for
me and others to wrap our heads around the things you have done,
so
don't be surprised if you get a delayed response to this
announcement
:-)
Congratulations! Dave
On Fri, Dec 29 2023 at 01:42:16 AM +0100, christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > Hi all! > > I‘m very excited to announce a project today that we have been > working on over the past two years: The *TraceDebugger* [1] is
a
new
> back-in-time/time-travel/omniscient debugging tool for Squeak
that
> allows you to record past method activations and states during > execution and explore them later. > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > Metacellonew > baseline:'TraceDebugger'; > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > /"repository: > 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> 6.0"/ > get; > load. > > **What can it do? (Features)** > > - *Record all method activations and historic states:*
Normally
step
> through a program in the debugger while automatically
recording its
> execution. > - *Replay execution of a traced program:* Navigate through all > method invocations using the /*context tree*/ or the /*Step
Back/Step
> Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, > let‘s start all over again" situations). > - *Interact with historic states:* Inspect/explore snapshots
of
> objects or send them any message. > - *State-centric debugging using the ***/History
Explorer*/**:*
> Gather, explore, and visualize all changes to an
object/expression
> over the recorded time ("When did this
variable/collection/screenshot
> change?"). > - *Additional navigation tools* for searching and filtering
the
> context tree. > - *Focus on interactivity:* No hours of recording, no GBs of
mem
> consumption - at least for common small to medium programs. > - *UI resembles the classic Smalltalk debugger:* You'll find
your
> familiar stepping buttons, code browsing tools, inspectors,
and
> shortcuts - plus more. > > The TraceDebugger is a general-purpose tool and not tied to > particular domains. In the past months, we have successfully
used
it
> to understand several bugs and interaction patterns in the
Trunk
> (Morphic layout/rendering, compiler/decompiler, code
simulation,
> …). The tool is also self-supporting, so you can debug a > TraceDebugger from another TraceDebugger. :-) > > **What can‘t it do (yet)? (Limitations and future work)** > > - *High performance:* While (sufficiently) fast enough for
most
> small to medium workloads, tracing very compute- or
mem-intensive
> operations may require more time (ex.: compiler/decompiler > invocation: <1s, HTTPS request: <10s, tool building: <5m,
complex
> rendering: minutes up to hours). > - *Not a dataflow analyzer:* The TraceDebugger does not track > dataflow events (e.g., argument passing) but only state
changes.
> - *No tracing of external states/events* for FFI/OSProcess or
custom
> VM modules. > - *No support for advanced language concepts* such as identity > forwarding/write barriers. > > **How does it work? (Implementation)** > > In one sentence: To record message sends and side effects, we > decorate the execution of certain bytecodes with tracing
extensions
> by modifying the code simulation using SimulationStudio [2]. > > In one paragraph: The program is executed in a specialized
code
> simulator that overrides instructions for sending messages
(e.g.,
> send, superSend) and for performing side-effects (e.g.,
popIntoRcvr,
> primitiveAtPut, push). All message sends are recorded in a
tree and
> all changed object slots are stored in a sparse time-dependent
memory
> structure before they are overwritten. For time-traveling, the
tree
> is traversed using a cursor. For accessing historic objects, a
proxy
> evaluates all messages sent to an object in another
specialized
> simulator (retracing simulator) that emulates historic states
for
the
> requested point in time by forwarding read primitives (e.g., > pushRcvr, primitiveAt) to the recorded memory. For gathering
state
> changes in the History Explorer efficiently, the query is
evaluated
> in a range retracing simulator with vectorization and fork
semantics.
> > In academic terms: We have published two papers about the > TraceDebugger that provide further details about its
implementation
> and its applications for program exploration, "Object-Centric > Time-Travel Debugging: Exploring Traces of Objects" [3] and > "Time-Awareness in Object Exploration Tools: Toward In Situ > Omniscient Debugging" [4]. > > In Smalltalk: Just check out the code base and explore it by > yourself! The class comments in TraceDebugger
> and TDBCursor code://TDBCursor should provide good starting
points.
> > **How can I use it?** > > Please try it out and report feedback! The TraceDebugger
supports
> the latest Squeak Trunk and Squeak 6.0. You can either
download a
> prepared all-in-one bundle on GitHub: > > https://github.com/hpi-swa-lab/squeak-tracedebugger/releases > > Or you can install it into your own image using Metacello: > > Metacellonew > baseline:'TraceDebugger'; > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > /"repository: > 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> 6.0"/ > get; > load. > > To get started, just open a normal debugger (e.g., by
selecting an
> expression and pressing Cmd+Shift+D to debug it) and then
press the
> "Trace It" button on the right. There‘s also some pretty
detailed
> documentation in the Help Browser <code:// TraceDebugger
showHelp>
> that covers everything you should know. > > My goal is to improve convenience and provide a useful tool
for the
> community, so I‘m very excited to hear your impressions,
ideas, and
> thoughts. Here, on GitHub, or in a private message. Let‘s have
a
> great discussion! :-) > > Best, > > Christoph (and Marcel) > > PS: Props to Eliot who brought up the original idea of
"subclassing
> from Context" for other reasons four years ago. [5] > > [1] https://github.com/hpi-swa-lab/squeak-tracedebugger > [2] https://github.com/LinqLover/SimulationStudio > [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
> Object-Centric Time-Travel Debugging: Exploring Traces of
Objects.
> https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
> of the 7th International Conference on the Art, Science, and > Engineering of Programming/ (/<Programming>'23 Companion/),
March
> 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages.
DOI:
> 10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
> PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 > [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
> Time-Awareness in Object Exploration Tools: Toward In Situ
Omniscient
> Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In > /Proceedings of the 2023 ACM SIGPLAN International Symposium
on New
> Ideas, New Paradigms, and Reflections on Programming and
Software/
> (/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM,
New
> York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > https://doi.org/10.1145/3622758.3622892. PDF: > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > [5] >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > --- > /Sent from//Squeak Inbox Talk > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Jaromir,
In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
The TraceDebugger now honors the suspended/terminated state of the debugged process so you cannot step beyond Processor activeProcess suspend any longer, like in a regular debugger.
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2024-01-02T19:53:31+01:00, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me. But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: Context runSimulated: [Context runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate Simulator debug: [thisContext findNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContext findNextHandlerContextStarting] debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContext class] debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulator debug: [thisContext findNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Example: do step through to the [^2] block and then step through again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to the traditional one?
In other words: if I run the traditional debugger and the Trace one side by side, should they display analogous steps?
In the other message I wrote about a different way to simulate primitive calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net> wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would be the best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would help tremendously to be able to ask right away instead of trudging through the code/help :) The code usually helps to understand **how** things work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new functionalities "step-by-step", and not be overwhelmed by all the might of the call tree :) Being able to go back is already a hell of an improvement! Or when you open an inspector on a context the context's state gets frozen in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the primitive call. The Trace debugger, however, starts executing the fallback code of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me. :-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is not ready for all your clever investigations regarding non-local returns and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards programs with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single global tree - its structure would change over the execution time, and when selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack at the viewed point in time (see also the '@ <timeIndex>' in the window title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You would notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to step out of #terminateTo: again. (You can update the TraceDebugger from the window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step further and eventually return back to the starting point. :-) If you want to, you can also turn off the preference "Show call tree in TraceDebugger" to make the TraceDebugger look more like a normal debugger, which also solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it removes one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the TraceDebugger. Better use it to explore how the simulator works. :-) For example, you could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method, press Cmd + f(ind), and type "return:from:" to investigate the behavior of your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort some things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it
stops
at Context>>terminate and the view gets corrupted (the initial part
of
the trace is hidden and can't be made visible unless clicking on
some of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of error:
Maybe this is just an unfortunate example... Or maybe I'm just doing something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Thanks for the reply, Dave! I will try to post one or two concrete
use
cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote:
> This sounds like really interesting work! I love the idea of
being
> able to interactively go back in "oops, I‘ve stepped too far,
let‘s
> start all over again" situations. It will probably take some
time for
> me and others to wrap our heads around the things you have done,
so
> don't be surprised if you get a delayed response to this
announcement
> :-) > > Congratulations! > Dave > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > Hi all! > > > > I‘m very excited to announce a project today that we have been > > working on over the past two years: The *TraceDebugger* [1] is
a
new > > back-in-time/time-travel/omniscient debugging tool for Squeak
that
> > allows you to record past method activations and states during > > execution and explore them later. > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > Metacellonew > > baseline:'TraceDebugger'; > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > /"repository: > > 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > 6.0"/ > > get; > > load. > > > > **What can it do? (Features)** > > > > - *Record all method activations and historic states:*
Normally
step > > through a program in the debugger while automatically
recording its
> > execution. > > - *Replay execution of a traced program:* Navigate through all > > method invocations using the /*context tree*/ or the /*Step Back/Step > > Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, > > let‘s start all over again" situations). > > - *Interact with historic states:* Inspect/explore snapshots
of
> > objects or send them any message. > > - *State-centric debugging using the ***/History
Explorer*/**:*
> > Gather, explore, and visualize all changes to an
object/expression
> > over the recorded time ("When did this variable/collection/screenshot > > change?"). > > - *Additional navigation tools* for searching and filtering
the
> > context tree. > > - *Focus on interactivity:* No hours of recording, no GBs of
mem
> > consumption - at least for common small to medium programs. > > - *UI resembles the classic Smalltalk debugger:* You'll find
your
> > familiar stepping buttons, code browsing tools, inspectors,
and
> > shortcuts - plus more. > > > > The TraceDebugger is a general-purpose tool and not tied to > > particular domains. In the past months, we have successfully
used
it > > to understand several bugs and interaction patterns in the
Trunk
> > (Morphic layout/rendering, compiler/decompiler, code
simulation,
> > …). The tool is also self-supporting, so you can debug a > > TraceDebugger from another TraceDebugger. :-) > > > > **What can‘t it do (yet)? (Limitations and future work)** > > > > - *High performance:* While (sufficiently) fast enough for
most
> > small to medium workloads, tracing very compute- or
mem-intensive
> > operations may require more time (ex.: compiler/decompiler > > invocation: <1s, HTTPS request: <10s, tool building: <5m,
complex
> > rendering: minutes up to hours). > > - *Not a dataflow analyzer:* The TraceDebugger does not track > > dataflow events (e.g., argument passing) but only state
changes.
> > - *No tracing of external states/events* for FFI/OSProcess or custom > > VM modules. > > - *No support for advanced language concepts* such as identity > > forwarding/write barriers. > > > > **How does it work? (Implementation)** > > > > In one sentence: To record message sends and side effects, we > > decorate the execution of certain bytecodes with tracing
extensions
> > by modifying the code simulation using SimulationStudio [2]. > > > > In one paragraph: The program is executed in a specialized
code
> > simulator that overrides instructions for sending messages
(e.g.,
> > send, superSend) and for performing side-effects (e.g., popIntoRcvr, > > primitiveAtPut, push). All message sends are recorded in a
tree and
> > all changed object slots are stored in a sparse time-dependent memory > > structure before they are overwritten. For time-traveling, the
tree
> > is traversed using a cursor. For accessing historic objects, a proxy > > evaluates all messages sent to an object in another
specialized
> > simulator (retracing simulator) that emulates historic states
for
the > > requested point in time by forwarding read primitives (e.g., > > pushRcvr, primitiveAt) to the recorded memory. For gathering
state
> > changes in the History Explorer efficiently, the query is
evaluated
> > in a range retracing simulator with vectorization and fork semantics. > > > > In academic terms: We have published two papers about the > > TraceDebugger that provide further details about its
implementation
> > and its applications for program exploration, "Object-Centric > > Time-Travel Debugging: Exploring Traces of Objects" [3] and > > "Time-Awareness in Object Exploration Tools: Toward In Situ > > Omniscient Debugging" [4]. > > > > In Smalltalk: Just check out the code base and explore it by > > yourself! The class comments in TraceDebugger code://TraceDebugger > > and TDBCursor code://TDBCursor should provide good starting points. > > > > **How can I use it?** > > > > Please try it out and report feedback! The TraceDebugger
supports
> > the latest Squeak Trunk and Squeak 6.0. You can either
download a
> > prepared all-in-one bundle on GitHub: > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger/releases > > > > Or you can install it into your own image using Metacello: > > > > Metacellonew > > baseline:'TraceDebugger'; > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > /"repository: > > 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > 6.0"/ > > get; > > load. > > > > To get started, just open a normal debugger (e.g., by
selecting an
> > expression and pressing Cmd+Shift+D to debug it) and then
press the
> > "Trace It" button on the right. There‘s also some pretty
detailed
> > documentation in the Help Browser <code:// TraceDebugger
showHelp>
> > that covers everything you should know. > > > > My goal is to improve convenience and provide a useful tool
for the
> > community, so I‘m very excited to hear your impressions,
ideas, and
> > thoughts. Here, on GitHub, or in a private message. Let‘s have
a
> > great discussion! :-) > > > > Best, > > > > Christoph (and Marcel) > > > > PS: Props to Eliot who brought up the original idea of
"subclassing
> > from Context" for other reasons four years ago. [5] > > > > [1] https://github.com/hpi-swa-lab/squeak-tracedebugger > > [2] https://github.com/LinqLover/SimulationStudio > > [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
> > Object-Centric Time-Travel Debugging: Exploring Traces of
Objects.
> > https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
> > of the 7th International Conference on the Art, Science, and > > Engineering of Programming/ (/<Programming>'23 Companion/),
March
> > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages.
DOI:
> > 10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
> > PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 > > [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
> > Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient > > Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In > > /Proceedings of the 2023 ACM SIGPLAN International Symposium
on New
> > Ideas, New Paradigms, and Reflections on Programming and
Software/
> > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM,
New
> > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > https://doi.org/10.1145/3622758.3622892. PDF: > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > [5] > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > --- > > /Sent from//Squeak Inbox Talk > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Christoph,
On 02-Jan-24 9:28:51 PM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger?
Sorry, that must have been an image before the latest update. In a fresh, latest image it works as you describe.
However - and forgive me if this is some kind of problem at my end too - I'm observing this:
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
[[self halt. ^ 1] on: BlockCannotReturn do: [:ex | ex resume] ] fork
The exact steps are: 1. do it in workspace 2. select Doit context (second from top) 3. switch to Trace debugger 4. click step Over
and this just happened:
The weird thing is next time I open the same image again, nothing bad can happen many times but all of a sudden it happens again.
A few times I also saw this weird error pop up:
In this case I used another example: [[true ifTrue: [self halt. ^ 1]] on: BlockCannotReturn do: [:ex | ex resume] ] fork
What could cause such intermittent failures? I tried downloading a new image again but the same happens.
I hope I'm not wasting your time with something happening just at my end. The only irregular thing I noticed is it's difficult to install or update the baseline the same github cache issue I had when I installed Squot. The installation fails a few times before it goes through successfully.
Thanks! best, Jaromir
However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcesssuspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
The TraceDebugger now honors the suspended/terminated state of the debugged process so you cannot step beyond Processor activeProcesssuspend any longer, like in a regular debugger.
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-02T19:53:31+01:00, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded
Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have
their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me. But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in
the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I
right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in
the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have
been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: Context runSimulated: [Context runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:,
#terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate
Simulator debug: [thisContext findNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContext findNextHandlerContextStarting] debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContext class] debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulator debug: [thisContext findNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
In other words: if I run the traditional debugger and the Trace one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would
be the
best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would
help
tremendously to be able to ask right away instead of trudging
through
the code/help :) The code usually helps to understand **how**
things
work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new
functionalities
"step-by-step", and not be overwhelmed by all the might of the
call
tree :) Being able to go back is already a hell of an improvement!
Or
when you open an inspector on a context the context's state gets
frozen
in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the
primitive
call. The Trace debugger, however, starts executing the fallback
code
of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me.
:-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is
not
ready for all your clever investigations regarding non-local
returns
and unwinding. ;-) In fact, your example reveals another
limitation
that I forgot to mention in the announcement, which regards
programs
with irregular context switches - e.g., generators/coroutines,
but
also non-local returns through unwind contexts. This is because
the
TraceDebugger stores and displays all method invocations in a
tree,
but in the case of manual context switches, there is no single
global
tree - its structure would change over the execution time, and
when
selecting a method invocation, it is not even clear to what
parent
(sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack
at
the viewed point in time (see also the '@ <timeIndex>' in the
window
title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You
would
notice the same in a normal debugger if you turned off the
optional
primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to
step
out of #terminateTo: again. (You can update the TraceDebugger
from the
window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step
further
and eventually return back to the starting point. :-) If you want
to,
you can also turn off the preference "Show call tree in
TraceDebugger"
to make the TraceDebugger look more like a normal debugger, which
also
solves the context switches issue. But in general - unless you
are
debugging unwinding stuff - I would not recommend that as it
removes
one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the
TraceDebugger.
Better use it to explore how the simulator works. :-) For
example, you
could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method,
press
Cmd + f(ind), and type "return:from:" to investigate the behavior
of
your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort
some
things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing
your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've
been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`,
it
stops
at Context>>terminate and the view gets corrupted (the initial
part
of
the trace is hidden and can't be made visible unless clicking
on
some of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of
error:
Maybe this is just an unfortunate example... Or maybe I'm just
doing
something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
>Thanks for the reply, Dave! I will try to post one or two
concrete
use
>cases about the TraceDebugger in the next couple of days, so
stay
>tuned. :-) > >Best, >Christoph > >--- >Sent from Squeak Inbox Talk >https://github.com/hpi-swa-lab/squeak-inbox-talk > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote: > > > This sounds like really interesting work! I love the idea
of
being
> > able to interactively go back in "oops, I‘ve stepped too
far,
let‘s
> > start all over again" situations. It will probably take
some
time for
> > me and others to wrap our heads around the things you have
done,
so
> > don't be surprised if you get a delayed response to this
announcement
> > :-) > > > > Congratulations! > > Dave > > > > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > > Hi all! > > > > > > I‘m very excited to announce a project today that we
have been
> > > working on over the past two years: The *TraceDebugger*
[1] is
a
>new > > > back-in-time/time-travel/omniscient debugging tool for
Squeak
that
> > > allows you to record past method activations and states
during
> > > execution and explore them later. > > > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > > > Metacellonew > > > baseline:'TraceDebugger'; > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > /"repository: > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > > 6.0"/ > > > get; > > > load. > > > > > > **What can it do? (Features)** > > > > > > - *Record all method activations and historic states:*
Normally
>step > > > through a program in the debugger while automatically
recording its
> > > execution. > > > - *Replay execution of a traced program:* Navigate
through all
> > > method invocations using the /*context tree*/ or the
/*Step
>Back/Step > > > Forward*/ buttons (to avoid these "oops, I‘ve stepped
too far,
> > > let‘s start all over again" situations). > > > - *Interact with historic states:* Inspect/explore
snapshots
of
> > > objects or send them any message. > > > - *State-centric debugging using the ***/History
Explorer*/**:*
> > > Gather, explore, and visualize all changes to an
object/expression
> > > over the recorded time ("When did this >variable/collection/screenshot > > > change?"). > > > - *Additional navigation tools* for searching and
filtering
the
> > > context tree. > > > - *Focus on interactivity:* No hours of recording, no
GBs of
mem
> > > consumption - at least for common small to medium
programs.
> > > - *UI resembles the classic Smalltalk debugger:* You'll
find
your
> > > familiar stepping buttons, code browsing tools,
inspectors,
and
> > > shortcuts - plus more. > > > > > > The TraceDebugger is a general-purpose tool and not tied
to
> > > particular domains. In the past months, we have
successfully
used
>it > > > to understand several bugs and interaction patterns in
the
Trunk
> > > (Morphic layout/rendering, compiler/decompiler, code
simulation,
> > > …). The tool is also self-supporting, so you can debug a > > > TraceDebugger from another TraceDebugger. :-) > > > > > > **What can‘t it do (yet)? (Limitations and future
work)**
> > > > > > - *High performance:* While (sufficiently) fast enough
for
most
> > > small to medium workloads, tracing very compute- or
mem-intensive
> > > operations may require more time (ex.:
compiler/decompiler
> > > invocation: <1s, HTTPS request: <10s, tool building:
<5m,
complex
> > > rendering: minutes up to hours). > > > - *Not a dataflow analyzer:* The TraceDebugger does not
track
> > > dataflow events (e.g., argument passing) but only state
changes.
> > > - *No tracing of external states/events* for
FFI/OSProcess or
>custom > > > VM modules. > > > - *No support for advanced language concepts* such as
identity
> > > forwarding/write barriers. > > > > > > **How does it work? (Implementation)** > > > > > > In one sentence: To record message sends and side
effects, we
> > > decorate the execution of certain bytecodes with tracing
extensions
> > > by modifying the code simulation using SimulationStudio
[2].
> > > > > > In one paragraph: The program is executed in a
specialized
code
> > > simulator that overrides instructions for sending
messages
(e.g.,
> > > send, superSend) and for performing side-effects (e.g., >popIntoRcvr, > > > primitiveAtPut, push). All message sends are recorded in
a
tree and
> > > all changed object slots are stored in a sparse
time-dependent
>memory > > > structure before they are overwritten. For
time-traveling, the
tree
> > > is traversed using a cursor. For accessing historic
objects, a
>proxy > > > evaluates all messages sent to an object in another
specialized
> > > simulator (retracing simulator) that emulates historic
states
for
>the > > > requested point in time by forwarding read primitives
(e.g.,
> > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
state
> > > changes in the History Explorer efficiently, the query
is
evaluated
> > > in a range retracing simulator with vectorization and
fork
>semantics. > > > > > > In academic terms: We have published two papers about
the
> > > TraceDebugger that provide further details about its
implementation
> > > and its applications for program exploration,
"Object-Centric
> > > Time-Travel Debugging: Exploring Traces of Objects" [3]
and
> > > "Time-Awareness in Object Exploration Tools: Toward In
Situ
> > > Omniscient Debugging" [4]. > > > > > > In Smalltalk: Just check out the code base and explore
it by
> > > yourself! The class comments in TraceDebugger >code://TraceDebugger > > > and TDBCursor code://TDBCursor should provide good
starting
>points. > > > > > > **How can I use it?** > > > > > > Please try it out and report feedback! The TraceDebugger
supports
> > > the latest Squeak Trunk and Squeak 6.0. You can either
download a
> > > prepared all-in-one bundle on GitHub: > > > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> > > > > > Or you can install it into your own image using
Metacello:
> > > > > > Metacellonew > > > baseline:'TraceDebugger'; > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > /"repository: > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > > 6.0"/ > > > get; > > > load. > > > > > > To get started, just open a normal debugger (e.g., by
selecting an
> > > expression and pressing Cmd+Shift+D to debug it) and
then
press the
> > > "Trace It" button on the right. There‘s also some pretty
detailed
> > > documentation in the Help Browser <code:// TraceDebugger
showHelp>
> > > that covers everything you should know. > > > > > > My goal is to improve convenience and provide a useful
tool
for the
> > > community, so I‘m very excited to hear your impressions,
ideas, and
> > > thoughts. Here, on GitHub, or in a private message.
Let‘s have
a
> > > great discussion! :-) > > > > > > Best, > > > > > > Christoph (and Marcel) > > > > > > PS: Props to Eliot who brought up the original idea of
"subclassing
> > > from Context" for other reasons four years ago. [5] > > > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
> > > [2] https://github.com/LinqLover/SimulationStudio > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > Object-Centric Time-Travel Debugging: Exploring Traces
of
Objects.
> > > https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
> > > of the 7th International Conference on the Art, Science,
and
> > > Engineering of Programming/ (/<Programming>'23
Companion/),
March
> > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
DOI:
> > > 10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
> > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
> > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > Time-Awareness in Object Exploration Tools: Toward In
Situ
>Omniscient > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
on New
> > > Ideas, New Paradigms, and Reflections on Programming and
Software/
> > > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal.
ACM,
New
> > > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > > https://doi.org/10.1145/3622758.3622892. PDF: > > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > > [5] > > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > > > --- > > > /Sent from//Squeak Inbox Talk > > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Jaromir,
very weird ... I was able to reproduce the sporadic stack dump with your second example, and in one situation I also got another error (I forgot to take a screenshot but it said something like errorSubscriptBounds on a BlockCannotReturn instance).
I think this is either a bug in the VM or the TraceDebugger is abusing a set of primitives (see the comment in Sandbox2>>#context:doPrimitiveNew:receiver:args: for a similar situation). It definitely will be hard to debug. How does your crash dump look like (if the VM has created one)? (Note that the latest stack always is appended to that file.) If you could collect a few SqueakDebug.logs and crash dumps, maybe that would help. I will try the same.
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
By the way, I think we're at 22933. Note that the files on files.squeak.org are only updated once a day (provided that nothing goes wrong).
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2024-01-03T23:19:53+00:00, mail@jaromir.net wrote:
Hi Christoph,
On 02-Jan-24 9:28:51 PM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger?
Sorry, that must have been an image before the latest update. In a fresh, latest image it works as you describe.
However - and forgive me if this is some kind of problem at my end too - I'm observing this:
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
[[self halt. ^ 1] on: BlockCannotReturn do: [:ex | ex resume] ] fork
The exact steps are:
- do it in workspace
- select Doit context (second from top)
- switch to Trace debugger
- click step Over
and this just happened:
The weird thing is next time I open the same image again, nothing bad can happen many times but all of a sudden it happens again.
A few times I also saw this weird error pop up:
In this case I used another example: [[true ifTrue: [self halt. ^ 1]] on: BlockCannotReturn do: [:ex | ex resume] ] fork
What could cause such intermittent failures? I tried downloading a new image again but the same happens.
I hope I'm not wasting your time with something happening just at my end. The only irregular thing I noticed is it's difficult to install or update the baseline the same github cache issue I had when I installed Squot. The installation fails a few times before it goes through successfully.
Thanks! best, Jaromir
However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcesssuspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
The TraceDebugger now honors the suspended/terminated state of the debugged process so you cannot step beyond Processor activeProcesssuspend any longer, like in a regular debugger.
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-02T19:53:31+01:00, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded
Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have
their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me. But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in
the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I
right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in
the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have
been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: Context runSimulated: [Context runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:,
#terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate
Simulator debug: [thisContext findNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContext findNextHandlerContextStarting] debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContext class] debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulator debug: [thisContext findNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
In other words: if I run the traditional debugger and the Trace one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would
be the
best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would
help
tremendously to be able to ask right away instead of trudging
through
the code/help :) The code usually helps to understand **how**
things
work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new
functionalities
"step-by-step", and not be overwhelmed by all the might of the
call
tree :) Being able to go back is already a hell of an improvement!
Or
when you open an inspector on a context the context's state gets
frozen
in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the
primitive
call. The Trace debugger, however, starts executing the fallback
code
of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me.
:-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is
not
ready for all your clever investigations regarding non-local
returns
and unwinding. ;-) In fact, your example reveals another
limitation
that I forgot to mention in the announcement, which regards
programs
with irregular context switches - e.g., generators/coroutines,
but
also non-local returns through unwind contexts. This is because
the
TraceDebugger stores and displays all method invocations in a
tree,
but in the case of manual context switches, there is no single
global
tree - its structure would change over the execution time, and
when
selecting a method invocation, it is not even clear to what
parent
(sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack
at
the viewed point in time (see also the '@ <timeIndex>' in the
window
title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You
would
notice the same in a normal debugger if you turned off the
optional
primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to
step
out of #terminateTo: again. (You can update the TraceDebugger
from the
window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step
further
and eventually return back to the starting point. :-) If you want
to,
you can also turn off the preference "Show call tree in
TraceDebugger"
to make the TraceDebugger look more like a normal debugger, which
also
solves the context switches issue. But in general - unless you
are
debugging unwinding stuff - I would not recommend that as it
removes
one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the
TraceDebugger.
Better use it to explore how the simulator works. :-) For
example, you
could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method,
press
Cmd + f(ind), and type "return:from:" to investigate the behavior
of
your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort
some
things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
> Hi Christoph, > > This indeed sounds like a GREAT idea! I look forward to seeing
your
use > cases to build the right intuition. > > In the meantime I've tried to debug/trace this example I've
been
working > with lately: > > [^2] ensure: [] > > If I start the debugger, hit `trace it` and then `step over`,
it
stops > at Context>>terminate and the view gets corrupted (the initial
part
of > the trace is hidden and can't be made visible unless clicking
on
some of > the pink lines - but not every line does it...) > > > > > If I then continue stepping over it ends up with some kind of
error:
> > > Maybe this is just an unfortunate example... Or maybe I'm just
doing
> something wrong... > > At any rate - THANKS for your effort!! > > > On 30-Dec-23 4:37:28 PM, christoph.thiede(a)student.hpi.uni-potsdam.de > wrote: > > >Thanks for the reply, Dave! I will try to post one or two
concrete
use > >cases about the TraceDebugger in the next couple of days, so
stay
> >tuned. :-) > > > >Best, > >Christoph > > > >--- > >Sent from Squeak Inbox Talk > >https://github.com/hpi-swa-lab/squeak-inbox-talk > > > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote: > > > > > This sounds like really interesting work! I love the idea
of
being > > > able to interactively go back in "oops, I‘ve stepped too
far,
let‘s > > > start all over again" situations. It will probably take
some
time for > > > me and others to wrap our heads around the things you have
done,
so > > > don't be surprised if you get a delayed response to this announcement > > > :-) > > > > > > Congratulations! > > > Dave > > > > > > > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > > > Hi all! > > > > > > > > I‘m very excited to announce a project today that we
have been
> > > > working on over the past two years: The *TraceDebugger*
[1] is
a > >new > > > > back-in-time/time-travel/omniscient debugging tool for
Squeak
that > > > > allows you to record past method activations and states
during
> > > > execution and explore them later. > > > > > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > > > > > Metacellonew > > > > baseline:'TraceDebugger'; > > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > > /"repository: > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak > > > > 6.0"/ > > > > get; > > > > load. > > > > > > > > **What can it do? (Features)** > > > > > > > > - *Record all method activations and historic states:* Normally > >step > > > > through a program in the debugger while automatically recording its > > > > execution. > > > > - *Replay execution of a traced program:* Navigate
through all
> > > > method invocations using the /*context tree*/ or the
/*Step
> >Back/Step > > > > Forward*/ buttons (to avoid these "oops, I‘ve stepped
too far,
> > > > let‘s start all over again" situations). > > > > - *Interact with historic states:* Inspect/explore
snapshots
of > > > > objects or send them any message. > > > > - *State-centric debugging using the ***/History Explorer*/**:* > > > > Gather, explore, and visualize all changes to an object/expression > > > > over the recorded time ("When did this > >variable/collection/screenshot > > > > change?"). > > > > - *Additional navigation tools* for searching and
filtering
the > > > > context tree. > > > > - *Focus on interactivity:* No hours of recording, no
GBs of
mem > > > > consumption - at least for common small to medium
programs.
> > > > - *UI resembles the classic Smalltalk debugger:* You'll
find
your > > > > familiar stepping buttons, code browsing tools,
inspectors,
and > > > > shortcuts - plus more. > > > > > > > > The TraceDebugger is a general-purpose tool and not tied
to
> > > > particular domains. In the past months, we have
successfully
used > >it > > > > to understand several bugs and interaction patterns in
the
Trunk > > > > (Morphic layout/rendering, compiler/decompiler, code simulation, > > > > …). The tool is also self-supporting, so you can debug a > > > > TraceDebugger from another TraceDebugger. :-) > > > > > > > > **What can‘t it do (yet)? (Limitations and future
work)**
> > > > > > > > - *High performance:* While (sufficiently) fast enough
for
most > > > > small to medium workloads, tracing very compute- or mem-intensive > > > > operations may require more time (ex.:
compiler/decompiler
> > > > invocation: <1s, HTTPS request: <10s, tool building:
<5m,
complex > > > > rendering: minutes up to hours). > > > > - *Not a dataflow analyzer:* The TraceDebugger does not
track
> > > > dataflow events (e.g., argument passing) but only state changes. > > > > - *No tracing of external states/events* for
FFI/OSProcess or
> >custom > > > > VM modules. > > > > - *No support for advanced language concepts* such as
identity
> > > > forwarding/write barriers. > > > > > > > > **How does it work? (Implementation)** > > > > > > > > In one sentence: To record message sends and side
effects, we
> > > > decorate the execution of certain bytecodes with tracing extensions > > > > by modifying the code simulation using SimulationStudio
[2].
> > > > > > > > In one paragraph: The program is executed in a
specialized
code > > > > simulator that overrides instructions for sending
messages
(e.g., > > > > send, superSend) and for performing side-effects (e.g., > >popIntoRcvr, > > > > primitiveAtPut, push). All message sends are recorded in
a
tree and > > > > all changed object slots are stored in a sparse
time-dependent
> >memory > > > > structure before they are overwritten. For
time-traveling, the
tree > > > > is traversed using a cursor. For accessing historic
objects, a
> >proxy > > > > evaluates all messages sent to an object in another specialized > > > > simulator (retracing simulator) that emulates historic
states
for > >the > > > > requested point in time by forwarding read primitives
(e.g.,
> > > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
state > > > > changes in the History Explorer efficiently, the query
is
evaluated > > > > in a range retracing simulator with vectorization and
fork
> >semantics. > > > > > > > > In academic terms: We have published two papers about
the
> > > > TraceDebugger that provide further details about its implementation > > > > and its applications for program exploration,
"Object-Centric
> > > > Time-Travel Debugging: Exploring Traces of Objects" [3]
and
> > > > "Time-Awareness in Object Exploration Tools: Toward In
Situ
> > > > Omniscient Debugging" [4]. > > > > > > > > In Smalltalk: Just check out the code base and explore
it by
> > > > yourself! The class comments in TraceDebugger > >code://TraceDebugger > > > > and TDBCursor code://TDBCursor should provide good
starting
> >points. > > > > > > > > **How can I use it?** > > > > > > > > Please try it out and report feedback! The TraceDebugger supports > > > > the latest Squeak Trunk and Squeak 6.0. You can either download a > > > > prepared all-in-one bundle on GitHub: > > > > > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> > > > > > > > Or you can install it into your own image using
Metacello:
> > > > > > > > Metacellonew > > > > baseline:'TraceDebugger'; > > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > > /"repository: > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak > > > > 6.0"/ > > > > get; > > > > load. > > > > > > > > To get started, just open a normal debugger (e.g., by selecting an > > > > expression and pressing Cmd+Shift+D to debug it) and
then
press the > > > > "Trace It" button on the right. There‘s also some pretty detailed > > > > documentation in the Help Browser <code:// TraceDebugger showHelp> > > > > that covers everything you should know. > > > > > > > > My goal is to improve convenience and provide a useful
tool
for the > > > > community, so I‘m very excited to hear your impressions, ideas, and > > > > thoughts. Here, on GitHub, or in a private message.
Let‘s have
a > > > > great discussion! :-) > > > > > > > > Best, > > > > > > > > Christoph (and Marcel) > > > > > > > > PS: Props to Eliot who brought up the original idea of "subclassing > > > > from Context" for other reasons four years ago. [5] > > > > > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
> > > > [2] https://github.com/LinqLover/SimulationStudio > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > > Object-Centric Time-Travel Debugging: Exploring Traces
of
Objects. > > > > https://doi.org/10.1145/3594671.3594678 In /Companion Proceedings > > > > of the 7th International Conference on the Art, Science,
and
> > > > Engineering of Programming/ (/<Programming>'23
Companion/),
March > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
DOI: > > > > 10.1145/3594671.3594678 https://doi.org/10.1145/3594671.3594678. > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
> > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > > Time-Awareness in Object Exploration Tools: Toward In
Situ
> >Omniscient > > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> > > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
on New > > > > Ideas, New Paradigms, and Reflections on Programming and Software/ > > > > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal.
ACM,
New > > > > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > > > https://doi.org/10.1145/3622758.3622892. PDF: > > > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > > > [5] > > > > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > > > > > --- > > > > /Sent from//Squeak Inbox Talk > > > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Yes, I also got to reproduce your first screenshot. It seems to matter that I press "debug" in the debugger first and then select the second line in the stack ...
--- Sent from Squeak Inbox Talk
On 2024-01-04T01:11:15+01:00, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
very weird ... I was able to reproduce the sporadic stack dump with your second example, and in one situation I also got another error (I forgot to take a screenshot but it said something like errorSubscriptBounds on a BlockCannotReturn instance).
I think this is either a bug in the VM or the TraceDebugger is abusing a set of primitives (see the comment in Sandbox2>>#context:doPrimitiveNew:receiver:args: for a similar situation). It definitely will be hard to debug. How does your crash dump look like (if the VM has created one)? (Note that the latest stack always is appended to that file.) If you could collect a few SqueakDebug.logs and crash dumps, maybe that would help. I will try the same.
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
By the way, I think we're at 22933. Note that the files on files.squeak.org are only updated once a day (provided that nothing goes wrong).
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-03T23:19:53+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
On 02-Jan-24 9:28:51 PM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger?
Sorry, that must have been an image before the latest update. In a fresh, latest image it works as you describe.
However - and forgive me if this is some kind of problem at my end too - I'm observing this:
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
[[self halt. ^ 1] on: BlockCannotReturn do: [:ex | ex resume] ] fork
The exact steps are:
- do it in workspace
- select Doit context (second from top)
- switch to Trace debugger
- click step Over
and this just happened:
The weird thing is next time I open the same image again, nothing bad can happen many times but all of a sudden it happens again.
A few times I also saw this weird error pop up:
In this case I used another example: [[true ifTrue: [self halt. ^ 1]] on: BlockCannotReturn do: [:ex | ex resume] ] fork
What could cause such intermittent failures? I tried downloading a new image again but the same happens.
I hope I'm not wasting your time with something happening just at my end. The only irregular thing I noticed is it's difficult to install or update the baseline the same github cache issue I had when I installed Squot. The installation fails a few times before it goes through successfully.
Thanks! best, Jaromir
However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcesssuspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
The TraceDebugger now honors the suspended/terminated state of the debugged process so you cannot step beyond Processor activeProcesssuspend any longer, like in a regular debugger.
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-02T19:53:31+01:00, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded
Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have
their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me. But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in
the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I
right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in
the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have
been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: Context runSimulated: [Context runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:,
#terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate
Simulator debug: [thisContext findNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContext findNextHandlerContextStarting] debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContext class] debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulator debug: [thisContext findNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
In other words: if I run the traditional debugger and the Trace one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would
be the
best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would
help
tremendously to be able to ask right away instead of trudging
through
the code/help :) The code usually helps to understand **how**
things
work, the mechanics, but rarely **why**, the intentions.
> you can also turn off the preference "Show call tree in TraceDebugger"
Thanks, that helps to familiarize myself with the new
functionalities
"step-by-step", and not be overwhelmed by all the might of the
call
tree :) Being able to go back is already a hell of an improvement!
Or
when you open an inspector on a context the context's state gets
frozen
in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the
primitive
call. The Trace debugger, however, starts executing the fallback
code
of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
>Hi Jaromir, > >thanks a lot for trying it out!! Your feedback means a lot to me.
:-)
> >Indeed you hit a pretty unfortunate example. The TraceDebugger is
not
>ready for all your clever investigations regarding non-local
returns
>and unwinding. ;-) In fact, your example reveals another
limitation
>that I forgot to mention in the announcement, which regards
programs
>with irregular context switches - e.g., generators/coroutines,
but
>also non-local returns through unwind contexts. This is because
the
>TraceDebugger stores and displays all method invocations in a
tree,
>but in the case of manual context switches, there is no single
global
>tree - its structure would change over the execution time, and
when
>selecting a method invocation, it is not even clear to what
parent
>(sender) it would belong, as there might be multiple. The current >solution is to display the tree from the perspective of the stack
at
>the viewed point in time (see also the '@ <timeIndex>' in the
window
>title), so it looks corrupted while stepping through >Context>>#terminateTo: as the stack is being manipulated. (You
would
>notice the same in a normal debugger if you turned off the
optional
>primitive 196 in this method - for SimulationContexts this method >always uses the fallback code.) > >Nevertheless, I have pushed some changes that should allow you to
step
>out of #terminateTo: again. (You can update the TraceDebugger
from the
>window menu icon at the right top, like all of my tools.) At some >point there will no method be displayed, but you can just step
further
>and eventually return back to the starting point. :-) If you want
to,
>you can also turn off the preference "Show call tree in
TraceDebugger"
>to make the TraceDebugger look more like a normal debugger, which
also
>solves the context switches issue. But in general - unless you
are
>debugging unwinding stuff - I would not recommend that as it
removes
>one important strength of the TraceDebugger. :-) > >But again, this is really not a prime example for the
TraceDebugger.
>Better use it to explore how the simulator works. :-) For
example, you
>could do the following: > >[ContextTest debug: #testBlockCannotReturn] debugTrace. > >And in that trace debugger, you could select the start method,
press
>Cmd + f(ind), and type "return:from:" to investigate the behavior
of
>your solution there again, etc. > >Thanks for your comments! This was a good chance for me to sort
some
>things out! :-) > >Best, >Christoph > >--- >Sent from Squeak Inbox Talk >https://github.com/hpi-swa-lab/squeak-inbox-talk > >On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote: > > > Hi Christoph, > > > > This indeed sounds like a GREAT idea! I look forward to seeing
your
>use > > cases to build the right intuition. > > > > In the meantime I've tried to debug/trace this example I've
been
>working > > with lately: > > > > [^2] ensure: [] > > > > If I start the debugger, hit `trace it` and then `step over`,
it
>stops > > at Context>>terminate and the view gets corrupted (the initial
part
>of > > the trace is hidden and can't be made visible unless clicking
on
>some of > > the pink lines - but not every line does it...) > > > > > > > > > > If I then continue stepping over it ends up with some kind of
error:
> > > > > > Maybe this is just an unfortunate example... Or maybe I'm just
doing
> > something wrong... > > > > At any rate - THANKS for your effort!! > > > > > > On 30-Dec-23 4:37:28 PM, >christoph.thiede(a)student.hpi.uni-potsdam.de > > wrote: > > > > >Thanks for the reply, Dave! I will try to post one or two
concrete
>use > > >cases about the TraceDebugger in the next couple of days, so
stay
> > >tuned. :-) > > > > > >Best, > > >Christoph > > > > > >--- > > >Sent from Squeak Inbox Talk > > >https://github.com/hpi-swa-lab/squeak-inbox-talk > > > > > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote: > > > > > > > This sounds like really interesting work! I love the idea
of
>being > > > > able to interactively go back in "oops, I‘ve stepped too
far,
>let‘s > > > > start all over again" situations. It will probably take
some
>time for > > > > me and others to wrap our heads around the things you have
done,
>so > > > > don't be surprised if you get a delayed response to this >announcement > > > > :-) > > > > > > > > Congratulations! > > > > Dave > > > > > > > > > > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > > > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > > > > Hi all! > > > > > > > > > > I‘m very excited to announce a project today that we
have been
> > > > > working on over the past two years: The *TraceDebugger*
[1] is
>a > > >new > > > > > back-in-time/time-travel/omniscient debugging tool for
Squeak
>that > > > > > allows you to record past method activations and states
during
> > > > > execution and explore them later. > > > > > > > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > > > > > > > Metacellonew > > > > > baseline:'TraceDebugger'; > > > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > > > /"repository: > > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
>Squeak > > > > > 6.0"/ > > > > > get; > > > > > load. > > > > > > > > > > **What can it do? (Features)** > > > > > > > > > > - *Record all method activations and historic states:* >Normally > > >step > > > > > through a program in the debugger while automatically >recording its > > > > > execution. > > > > > - *Replay execution of a traced program:* Navigate
through all
> > > > > method invocations using the /*context tree*/ or the
/*Step
> > >Back/Step > > > > > Forward*/ buttons (to avoid these "oops, I‘ve stepped
too far,
> > > > > let‘s start all over again" situations). > > > > > - *Interact with historic states:* Inspect/explore
snapshots
>of > > > > > objects or send them any message. > > > > > - *State-centric debugging using the ***/History >Explorer*/**:* > > > > > Gather, explore, and visualize all changes to an >object/expression > > > > > over the recorded time ("When did this > > >variable/collection/screenshot > > > > > change?"). > > > > > - *Additional navigation tools* for searching and
filtering
>the > > > > > context tree. > > > > > - *Focus on interactivity:* No hours of recording, no
GBs of
>mem > > > > > consumption - at least for common small to medium
programs.
> > > > > - *UI resembles the classic Smalltalk debugger:* You'll
find
>your > > > > > familiar stepping buttons, code browsing tools,
inspectors,
>and > > > > > shortcuts - plus more. > > > > > > > > > > The TraceDebugger is a general-purpose tool and not tied
to
> > > > > particular domains. In the past months, we have
successfully
>used > > >it > > > > > to understand several bugs and interaction patterns in
the
>Trunk > > > > > (Morphic layout/rendering, compiler/decompiler, code >simulation, > > > > > …). The tool is also self-supporting, so you can debug a > > > > > TraceDebugger from another TraceDebugger. :-) > > > > > > > > > > **What can‘t it do (yet)? (Limitations and future
work)**
> > > > > > > > > > - *High performance:* While (sufficiently) fast enough
for
>most > > > > > small to medium workloads, tracing very compute- or >mem-intensive > > > > > operations may require more time (ex.:
compiler/decompiler
> > > > > invocation: <1s, HTTPS request: <10s, tool building:
<5m,
>complex > > > > > rendering: minutes up to hours). > > > > > - *Not a dataflow analyzer:* The TraceDebugger does not
track
> > > > > dataflow events (e.g., argument passing) but only state >changes. > > > > > - *No tracing of external states/events* for
FFI/OSProcess or
> > >custom > > > > > VM modules. > > > > > - *No support for advanced language concepts* such as
identity
> > > > > forwarding/write barriers. > > > > > > > > > > **How does it work? (Implementation)** > > > > > > > > > > In one sentence: To record message sends and side
effects, we
> > > > > decorate the execution of certain bytecodes with tracing >extensions > > > > > by modifying the code simulation using SimulationStudio
[2].
> > > > > > > > > > In one paragraph: The program is executed in a
specialized
>code > > > > > simulator that overrides instructions for sending
messages
>(e.g., > > > > > send, superSend) and for performing side-effects (e.g., > > >popIntoRcvr, > > > > > primitiveAtPut, push). All message sends are recorded in
a
>tree and > > > > > all changed object slots are stored in a sparse
time-dependent
> > >memory > > > > > structure before they are overwritten. For
time-traveling, the
>tree > > > > > is traversed using a cursor. For accessing historic
objects, a
> > >proxy > > > > > evaluates all messages sent to an object in another >specialized > > > > > simulator (retracing simulator) that emulates historic
states
>for > > >the > > > > > requested point in time by forwarding read primitives
(e.g.,
> > > > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
>state > > > > > changes in the History Explorer efficiently, the query
is
>evaluated > > > > > in a range retracing simulator with vectorization and
fork
> > >semantics. > > > > > > > > > > In academic terms: We have published two papers about
the
> > > > > TraceDebugger that provide further details about its >implementation > > > > > and its applications for program exploration,
"Object-Centric
> > > > > Time-Travel Debugging: Exploring Traces of Objects" [3]
and
> > > > > "Time-Awareness in Object Exploration Tools: Toward In
Situ
> > > > > Omniscient Debugging" [4]. > > > > > > > > > > In Smalltalk: Just check out the code base and explore
it by
> > > > > yourself! The class comments in TraceDebugger > > >code://TraceDebugger > > > > > and TDBCursor code://TDBCursor should provide good
starting
> > >points. > > > > > > > > > > **How can I use it?** > > > > > > > > > > Please try it out and report feedback! The TraceDebugger >supports > > > > > the latest Squeak Trunk and Squeak 6.0. You can either >download a > > > > > prepared all-in-one bundle on GitHub: > > > > > > > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> > > > > > > > > > Or you can install it into your own image using
Metacello:
> > > > > > > > > > Metacellonew > > > > > baseline:'TraceDebugger'; > > > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > > > /"repository: > > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
>Squeak > > > > > 6.0"/ > > > > > get; > > > > > load. > > > > > > > > > > To get started, just open a normal debugger (e.g., by >selecting an > > > > > expression and pressing Cmd+Shift+D to debug it) and
then
>press the > > > > > "Trace It" button on the right. There‘s also some pretty >detailed > > > > > documentation in the Help Browser <code:// TraceDebugger >showHelp> > > > > > that covers everything you should know. > > > > > > > > > > My goal is to improve convenience and provide a useful
tool
>for the > > > > > community, so I‘m very excited to hear your impressions, >ideas, and > > > > > thoughts. Here, on GitHub, or in a private message.
Let‘s have
>a > > > > > great discussion! :-) > > > > > > > > > > Best, > > > > > > > > > > Christoph (and Marcel) > > > > > > > > > > PS: Props to Eliot who brought up the original idea of >"subclassing > > > > > from Context" for other reasons four years ago. [5] > > > > > > > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
> > > > > [2] https://github.com/LinqLover/SimulationStudio > > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
>2023. > > > > > Object-Centric Time-Travel Debugging: Exploring Traces
of
>Objects. > > > > > https://doi.org/10.1145/3594671.3594678 In /Companion >Proceedings > > > > > of the 7th International Conference on the Art, Science,
and
> > > > > Engineering of Programming/ (/<Programming>'23
Companion/),
>March > > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
>DOI: > > > > > 10.1145/3594671.3594678 >https://doi.org/10.1145/3594671.3594678. > > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
> > > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
>2023. > > > > > Time-Awareness in Object Exploration Tools: Toward In
Situ
> > >Omniscient > > > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> > > > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
>on New > > > > > Ideas, New Paradigms, and Reflections on Programming and >Software/ > > > > > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal.
ACM,
>New > > > > > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > > > > https://doi.org/10.1145/3622758.3622892. PDF: > > > > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > > > > [5] > > > > > > > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > > > > > > > --- > > > > > /Sent from//Squeak Inbox Talk > > > > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
It's getting even weirder ... One of my crash dumps looks like this:
Smalltalk stack dump: 0000009f6e7d0058 I MessageCatcher class(Behavior)>name 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d0088 M MessageCatcher class(Class)>name 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d00c0 M MessageCatcher class(ClassDescription)>printOn: 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d00f8 M [] in MessageCatcher class>printStringLimitedTo: 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d0138 M String class(SequenceableCollection class)>streamContents:limitedTo: 00007ff6cc3ed0d8: a(n) String 0000009f6e7d0180 M MessageCatcher class(Object)>printStringLimitedTo: 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d01b8 M MessageCatcher class(Object)>printString 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d01e8 M MessageCatcher class(Object)>asString 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d0218 M MessageCatcher class(Object)>asStringOrText 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d0290 M LazyListMorph>display:atRow:on: 00007ff6d13792b8: a(n) LazyListMorph 0000009f6e7d0308 M LazyListMorph>drawOn: 00007ff6d13792b8: a(n) LazyListMorph 0000009f6e7dede0 M FormCanvas(Canvas)>draw: 00007ff6cbd4a258: a(n) FormCanvas 0000009f6e7dee18 M FormCanvas(Canvas)>drawMorph: 00007ff6cbd4a258: a(n) FormCanvas
But my image does not have any eventual user of MessageCatcher, just a couple of methods that are never sent ...
There seems something serious wrong with the object layout. I wish I knew how to debug this in VMMaker ...
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2024-01-04T01:17:30+01:00, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Yes, I also got to reproduce your first screenshot. It seems to matter that I press "debug" in the debugger first and then select the second line in the stack ...
Sent from Squeak Inbox Talk
On 2024-01-04T01:11:15+01:00, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
very weird ... I was able to reproduce the sporadic stack dump with your second example, and in one situation I also got another error (I forgot to take a screenshot but it said something like errorSubscriptBounds on a BlockCannotReturn instance).
I think this is either a bug in the VM or the TraceDebugger is abusing a set of primitives (see the comment in Sandbox2>>#context:doPrimitiveNew:receiver:args: for a similar situation). It definitely will be hard to debug. How does your crash dump look like (if the VM has created one)? (Note that the latest stack always is appended to that file.) If you could collect a few SqueakDebug.logs and crash dumps, maybe that would help. I will try the same.
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
By the way, I think we're at 22933. Note that the files on files.squeak.org are only updated once a day (provided that nothing goes wrong).
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-03T23:19:53+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
On 02-Jan-24 9:28:51 PM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger?
Sorry, that must have been an image before the latest update. In a fresh, latest image it works as you describe.
However - and forgive me if this is some kind of problem at my end too - I'm observing this:
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
[[self halt. ^ 1] on: BlockCannotReturn do: [:ex | ex resume] ] fork
The exact steps are:
- do it in workspace
- select Doit context (second from top)
- switch to Trace debugger
- click step Over
and this just happened:
The weird thing is next time I open the same image again, nothing bad can happen many times but all of a sudden it happens again.
A few times I also saw this weird error pop up:
In this case I used another example: [[true ifTrue: [self halt. ^ 1]] on: BlockCannotReturn do: [:ex | ex resume] ] fork
What could cause such intermittent failures? I tried downloading a new image again but the same happens.
I hope I'm not wasting your time with something happening just at my end. The only irregular thing I noticed is it's difficult to install or update the baseline the same github cache issue I had when I installed Squot. The installation fails a few times before it goes through successfully.
Thanks! best, Jaromir
However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcesssuspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
The TraceDebugger now honors the suspended/terminated state of the debugged process so you cannot step beyond Processor activeProcesssuspend any longer, like in a regular debugger.
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-02T19:53:31+01:00, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded
Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have
their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me. But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in
the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I
right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in
the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have
been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: Context runSimulated: [Context runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:,
#terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate
Simulator debug: [thisContext findNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContext findNextHandlerContextStarting] debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContext class] debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulator debug: [thisContext findNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
In other words: if I run the traditional debugger and the Trace one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
>Hi Christoph, > >Is it ok that I ask questions about the new debugger? What would
be the
>best format for such a "Q&A" - here or perhaps within a topic on >squeak-smalltalk/squeak-object-memory? I don't expect a flood of >questions but to get a bit familiar with your debugger it would
help
>tremendously to be able to ask right away instead of trudging
through
>the code/help :) The code usually helps to understand **how**
things
>work, the mechanics, but rarely **why**, the intentions. > > > you can also turn off the preference "Show call tree in >TraceDebugger" > >Thanks, that helps to familiarize myself with the new
functionalities
>"step-by-step", and not be overwhelmed by all the might of the
call
>tree :) Being able to go back is already a hell of an improvement!
Or
>when you open an inspector on a context the context's state gets
frozen
>in time and won't change when you proceed debugging - another >groundbreaking change! > >Question: >In the traditional debugger, when you step into a primitive, the >primitive gets executed and the simulation moves over the
primitive
>call. The Trace debugger, however, starts executing the fallback
code
>of the primitive call - why is that? > >Screenshot after step into #terminateTo: > > > >Thanks again, >Jaromir > >On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
>wrote: > >>Hi Jaromir, >> >>thanks a lot for trying it out!! Your feedback means a lot to me.
:-)
>> >>Indeed you hit a pretty unfortunate example. The TraceDebugger is
not
>>ready for all your clever investigations regarding non-local
returns
>>and unwinding. ;-) In fact, your example reveals another
limitation
>>that I forgot to mention in the announcement, which regards
programs
>>with irregular context switches - e.g., generators/coroutines,
but
>>also non-local returns through unwind contexts. This is because
the
>>TraceDebugger stores and displays all method invocations in a
tree,
>>but in the case of manual context switches, there is no single
global
>>tree - its structure would change over the execution time, and
when
>>selecting a method invocation, it is not even clear to what
parent
>>(sender) it would belong, as there might be multiple. The current >>solution is to display the tree from the perspective of the stack
at
>>the viewed point in time (see also the '@ <timeIndex>' in the
window
>>title), so it looks corrupted while stepping through >>Context>>#terminateTo: as the stack is being manipulated. (You
would
>>notice the same in a normal debugger if you turned off the
optional
>>primitive 196 in this method - for SimulationContexts this method >>always uses the fallback code.) >> >>Nevertheless, I have pushed some changes that should allow you to
step
>>out of #terminateTo: again. (You can update the TraceDebugger
from the
>>window menu icon at the right top, like all of my tools.) At some >>point there will no method be displayed, but you can just step
further
>>and eventually return back to the starting point. :-) If you want
to,
>>you can also turn off the preference "Show call tree in
TraceDebugger"
>>to make the TraceDebugger look more like a normal debugger, which
also
>>solves the context switches issue. But in general - unless you
are
>>debugging unwinding stuff - I would not recommend that as it
removes
>>one important strength of the TraceDebugger. :-) >> >>But again, this is really not a prime example for the
TraceDebugger.
>>Better use it to explore how the simulator works. :-) For
example, you
>>could do the following: >> >>[ContextTest debug: #testBlockCannotReturn] debugTrace. >> >>And in that trace debugger, you could select the start method,
press
>>Cmd + f(ind), and type "return:from:" to investigate the behavior
of
>>your solution there again, etc. >> >>Thanks for your comments! This was a good chance for me to sort
some
>>things out! :-) >> >>Best, >>Christoph >> >>--- >>Sent from Squeak Inbox Talk >>https://github.com/hpi-swa-lab/squeak-inbox-talk >> >>On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote: >> >> > Hi Christoph, >> > >> > This indeed sounds like a GREAT idea! I look forward to seeing
your
>>use >> > cases to build the right intuition. >> > >> > In the meantime I've tried to debug/trace this example I've
been
>>working >> > with lately: >> > >> > [^2] ensure: [] >> > >> > If I start the debugger, hit `trace it` and then `step over`,
it
>>stops >> > at Context>>terminate and the view gets corrupted (the initial
part
>>of >> > the trace is hidden and can't be made visible unless clicking
on
>>some of >> > the pink lines - but not every line does it...) >> > >> > >> > >> > >> > If I then continue stepping over it ends up with some kind of
error:
>> > >> > >> > Maybe this is just an unfortunate example... Or maybe I'm just
doing
>> > something wrong... >> > >> > At any rate - THANKS for your effort!! >> > >> > >> > On 30-Dec-23 4:37:28 PM, >>christoph.thiede(a)student.hpi.uni-potsdam.de >> > wrote: >> > >> > >Thanks for the reply, Dave! I will try to post one or two
concrete
>>use >> > >cases about the TraceDebugger in the next couple of days, so
stay
>> > >tuned. :-) >> > > >> > >Best, >> > >Christoph >> > > >> > >--- >> > >Sent from Squeak Inbox Talk >> > >https://github.com/hpi-swa-lab/squeak-inbox-talk >> > > >> > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote: >> > > >> > > > This sounds like really interesting work! I love the idea
of
>>being >> > > > able to interactively go back in "oops, I‘ve stepped too
far,
>>let‘s >> > > > start all over again" situations. It will probably take
some
>>time for >> > > > me and others to wrap our heads around the things you have
done,
>>so >> > > > don't be surprised if you get a delayed response to this >>announcement >> > > > :-) >> > > > >> > > > Congratulations! >> > > > Dave >> > > > >> > > > >> > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, >> > > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: >> > > > > Hi all! >> > > > > >> > > > > I‘m very excited to announce a project today that we
have been
>> > > > > working on over the past two years: The *TraceDebugger*
[1] is
>>a >> > >new >> > > > > back-in-time/time-travel/omniscient debugging tool for
Squeak
>>that >> > > > > allows you to record past method activations and states
during
>> > > > > execution and explore them later. >> > > > > >> > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger >> > > > > >> > > > > Metacellonew >> > > > > baseline:'TraceDebugger'; >> > > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; >> > > > > /"repository: >> > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
>>Squeak >> > > > > 6.0"/ >> > > > > get; >> > > > > load. >> > > > > >> > > > > **What can it do? (Features)** >> > > > > >> > > > > - *Record all method activations and historic states:* >>Normally >> > >step >> > > > > through a program in the debugger while automatically >>recording its >> > > > > execution. >> > > > > - *Replay execution of a traced program:* Navigate
through all
>> > > > > method invocations using the /*context tree*/ or the
/*Step
>> > >Back/Step >> > > > > Forward*/ buttons (to avoid these "oops, I‘ve stepped
too far,
>> > > > > let‘s start all over again" situations). >> > > > > - *Interact with historic states:* Inspect/explore
snapshots
>>of >> > > > > objects or send them any message. >> > > > > - *State-centric debugging using the ***/History >>Explorer*/**:* >> > > > > Gather, explore, and visualize all changes to an >>object/expression >> > > > > over the recorded time ("When did this >> > >variable/collection/screenshot >> > > > > change?"). >> > > > > - *Additional navigation tools* for searching and
filtering
>>the >> > > > > context tree. >> > > > > - *Focus on interactivity:* No hours of recording, no
GBs of
>>mem >> > > > > consumption - at least for common small to medium
programs.
>> > > > > - *UI resembles the classic Smalltalk debugger:* You'll
find
>>your >> > > > > familiar stepping buttons, code browsing tools,
inspectors,
>>and >> > > > > shortcuts - plus more. >> > > > > >> > > > > The TraceDebugger is a general-purpose tool and not tied
to
>> > > > > particular domains. In the past months, we have
successfully
>>used >> > >it >> > > > > to understand several bugs and interaction patterns in
the
>>Trunk >> > > > > (Morphic layout/rendering, compiler/decompiler, code >>simulation, >> > > > > …). The tool is also self-supporting, so you can debug a >> > > > > TraceDebugger from another TraceDebugger. :-) >> > > > > >> > > > > **What can‘t it do (yet)? (Limitations and future
work)**
>> > > > > >> > > > > - *High performance:* While (sufficiently) fast enough
for
>>most >> > > > > small to medium workloads, tracing very compute- or >>mem-intensive >> > > > > operations may require more time (ex.:
compiler/decompiler
>> > > > > invocation: <1s, HTTPS request: <10s, tool building:
<5m,
>>complex >> > > > > rendering: minutes up to hours). >> > > > > - *Not a dataflow analyzer:* The TraceDebugger does not
track
>> > > > > dataflow events (e.g., argument passing) but only state >>changes. >> > > > > - *No tracing of external states/events* for
FFI/OSProcess or
>> > >custom >> > > > > VM modules. >> > > > > - *No support for advanced language concepts* such as
identity
>> > > > > forwarding/write barriers. >> > > > > >> > > > > **How does it work? (Implementation)** >> > > > > >> > > > > In one sentence: To record message sends and side
effects, we
>> > > > > decorate the execution of certain bytecodes with tracing >>extensions >> > > > > by modifying the code simulation using SimulationStudio
[2].
>> > > > > >> > > > > In one paragraph: The program is executed in a
specialized
>>code >> > > > > simulator that overrides instructions for sending
messages
>>(e.g., >> > > > > send, superSend) and for performing side-effects (e.g., >> > >popIntoRcvr, >> > > > > primitiveAtPut, push). All message sends are recorded in
a
>>tree and >> > > > > all changed object slots are stored in a sparse
time-dependent
>> > >memory >> > > > > structure before they are overwritten. For
time-traveling, the
>>tree >> > > > > is traversed using a cursor. For accessing historic
objects, a
>> > >proxy >> > > > > evaluates all messages sent to an object in another >>specialized >> > > > > simulator (retracing simulator) that emulates historic
states
>>for >> > >the >> > > > > requested point in time by forwarding read primitives
(e.g.,
>> > > > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
>>state >> > > > > changes in the History Explorer efficiently, the query
is
>>evaluated >> > > > > in a range retracing simulator with vectorization and
fork
>> > >semantics. >> > > > > >> > > > > In academic terms: We have published two papers about
the
>> > > > > TraceDebugger that provide further details about its >>implementation >> > > > > and its applications for program exploration,
"Object-Centric
>> > > > > Time-Travel Debugging: Exploring Traces of Objects" [3]
and
>> > > > > "Time-Awareness in Object Exploration Tools: Toward In
Situ
>> > > > > Omniscient Debugging" [4]. >> > > > > >> > > > > In Smalltalk: Just check out the code base and explore
it by
>> > > > > yourself! The class comments in TraceDebugger >> > >code://TraceDebugger >> > > > > and TDBCursor code://TDBCursor should provide good
starting
>> > >points. >> > > > > >> > > > > **How can I use it?** >> > > > > >> > > > > Please try it out and report feedback! The TraceDebugger >>supports >> > > > > the latest Squeak Trunk and Squeak 6.0. You can either >>download a >> > > > > prepared all-in-one bundle on GitHub: >> > > > > >> > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
>> > > > > >> > > > > Or you can install it into your own image using
Metacello:
>> > > > > >> > > > > Metacellonew >> > > > > baseline:'TraceDebugger'; >> > > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; >> > > > > /"repository: >> > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
>>Squeak >> > > > > 6.0"/ >> > > > > get; >> > > > > load. >> > > > > >> > > > > To get started, just open a normal debugger (e.g., by >>selecting an >> > > > > expression and pressing Cmd+Shift+D to debug it) and
then
>>press the >> > > > > "Trace It" button on the right. There‘s also some pretty >>detailed >> > > > > documentation in the Help Browser <code:// TraceDebugger >>showHelp> >> > > > > that covers everything you should know. >> > > > > >> > > > > My goal is to improve convenience and provide a useful
tool
>>for the >> > > > > community, so I‘m very excited to hear your impressions, >>ideas, and >> > > > > thoughts. Here, on GitHub, or in a private message.
Let‘s have
>>a >> > > > > great discussion! :-) >> > > > > >> > > > > Best, >> > > > > >> > > > > Christoph (and Marcel) >> > > > > >> > > > > PS: Props to Eliot who brought up the original idea of >>"subclassing >> > > > > from Context" for other reasons four years ago. [5] >> > > > > >> > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
>> > > > > [2] https://github.com/LinqLover/SimulationStudio >> > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
>>2023. >> > > > > Object-Centric Time-Travel Debugging: Exploring Traces
of
>>Objects. >> > > > > https://doi.org/10.1145/3594671.3594678 In /Companion >>Proceedings >> > > > > of the 7th International Conference on the Art, Science,
and
>> > > > > Engineering of Programming/ (/<Programming>'23
Companion/),
>>March >> > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
>>DOI: >> > > > > 10.1145/3594671.3594678 >>https://doi.org/10.1145/3594671.3594678. >> > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
>> > > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
>>2023. >> > > > > Time-Awareness in Object Exploration Tools: Toward In
Situ
>> > >Omniscient >> > > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
>> > > > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
>>on New >> > > > > Ideas, New Paradigms, and Reflections on Programming and >>Software/ >> > > > > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal.
ACM,
>>New >> > > > > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 >> > > > > https://doi.org/10.1145/3622758.3622892. PDF: >> > > > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 >> > > > > [5] >> > > > > >> > >>
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
>> > > > > >> > > > > --- >> > > > > /Sent from//Squeak Inbox Talk >> > > > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Christoph,
I have no idea where to start looking... I've managed to catch a few logs but they seem to be incomplete, as if the VM died before being able to finish them (but my experience in this department is zero).
As for the other type of errors that show up from time to time, here's a screenshot:
What surprises me the `stack` variable contains a Process (or sometimes an Exception - BCR) at this point.
Another observation was the frequency of the crashes decreased when I restarted my PC and the allocated memory decreased. As it started increasing the frequency of the crashes increased - but I have no idea whether this may have anything to do with it. (I have 16GB and the system currently uses 9, so not even over the limit...).
There's also an older dump with a chinese name from Jan-2 I've found but I have no idea what that is :)))
That's it for the moment :) best, Jaromir
On 04-Jan-24 1:34:11 AM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
It's getting even weirder ... One of my crash dumps looks like this:
Smalltalk stack dump: 0000009f6e7d0058 I MessageCatcher class(Behavior)>name 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d0088 M MessageCatcher class(Class)>name 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d00c0 M MessageCatcher class(ClassDescription)>printOn: 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d00f8 M [] in MessageCatcher class>printStringLimitedTo: 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d0138 M String class(SequenceableCollection class)>streamContents:limitedTo: 00007ff6cc3ed0d8: a(n) String 0000009f6e7d0180 M MessageCatcher class(Object)>printStringLimitedTo: 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d01b8 M MessageCatcher class(Object)>printString 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d01e8 M MessageCatcher class(Object)>asString 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d0218 M MessageCatcher class(Object)>asStringOrText 00007ff6d1423ad0: a(n) MessageCatcher 0000009f6e7d0290 M LazyListMorph>display:atRow:on: 00007ff6d13792b8: a(n) LazyListMorph 0000009f6e7d0308 M LazyListMorph>drawOn: 00007ff6d13792b8: a(n) LazyListMorph 0000009f6e7dede0 M FormCanvas(Canvas)>draw: 00007ff6cbd4a258: a(n) FormCanvas 0000009f6e7dee18 M FormCanvas(Canvas)>drawMorph: 00007ff6cbd4a258: a(n) FormCanvas
But my image does not have any eventual user of MessageCatcher, just a couple of methods that are never sent ...
There seems something serious wrong with the object layout. I wish I knew how to debug this in VMMaker ...
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-04T01:17:30+01:00, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Yes, I also got to reproduce your first screenshot. It seems to
matter that I press "debug" in the debugger first and then select the second line in the stack ...
Sent from Squeak Inbox Talk
On 2024-01-04T01:11:15+01:00,
christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
very weird ... I was able to reproduce the sporadic stack dump with
your second example, and in one situation I also got another error (I forgot to take a screenshot but it said something like errorSubscriptBounds on a BlockCannotReturn instance).
I think this is either a bug in the VM or the TraceDebugger is
abusing a set of primitives (see the comment in Sandbox2>>#context:doPrimitiveNew:receiver:args: for a similar situation). It definitely will be hard to debug. How does your crash dump look like (if the VM has created one)? (Note that the latest stack always is appended to that file.) If you could collect a few SqueakDebug.logs and crash dumps, maybe that would help. I will try the same.
I downloaded a fresh image 22929, installed the Trace debugger,
turned off the tree view and tried to run this example:
By the way, I think we're at 22933. Note that the files on
files.squeak.org are only updated once a day (provided that nothing goes wrong).
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-03T23:19:53+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
On 02-Jan-24 9:28:51 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
> In the new Trace debugger you end up with the
#cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then
step
through again, I land in Context>>terminateTo:. Are you using
the
latest version of trunk and TraceDebugger?
Sorry, that must have been an image before the latest update. In
a
fresh, latest image it works as you describe.
However - and forgive me if this is some kind of problem at my
end too -
I'm observing this:
I downloaded a fresh image 22929, installed the Trace debugger,
turned
off the tree view and tried to run this example:
[[self halt. ^ 1] on: BlockCannotReturn do: [:ex | ex resume] ]
fork
The exact steps are:
- do it in workspace
- select Doit context (second from top)
- switch to Trace debugger
- click step Over
and this just happened:
The weird thing is next time I open the same image again, nothing
bad
can happen many times but all of a sudden it happens again.
A few times I also saw this weird error pop up:
In this case I used another example: [[true ifTrue: [self halt. ^ 1]] on: BlockCannotReturn do: [:ex |
ex
resume] ] fork
What could cause such intermittent failures? I tried downloading
a new
image again but the same happens.
I hope I'm not wasting your time with something happening just at
my
end. The only irregular thing I noticed is it's difficult to
install or
update the baseline the same github cache issue I had when I
installed
Squot. The installation fails a few times before it goes through successfully.
Thanks! best, Jaromir
However, you currently end up in #cannotReturn: when stepping
beyond
the Processor activeProcesssuspend in the bottom context of a
process
using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the
suspended/terminated
state of the interrupted process. Maybe it should ...
The TraceDebugger now honors the suspended/terminated state of
the
debugged process so you cannot step beyond Processor activeProcesssuspend any longer, like in a regular debugger.
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-02T19:53:31+01:00, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have
uploaded
Collections-ct.1061 to the inbox which would fix it. However, it
is a
bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so
maybe
this needs further discussion.
> Or when you open an inspector on a context the context's
state gets
frozen in time and won't change when you proceed debugging -
another
groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot
explorers") have
their pros and cons. As a con, I often found it inconvenient
that I
cannot watch the changing state of certain objects in extra
windows as
I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger
might be
confusing as well because there is no clear visual connection
... It's
an unsolved UX problem for me. But I'm glad they work well for
you. :-)
If you have any better ideas, let me know!
Regarding your questions about the behavior of code when
being run in
the TraceDebugger:
> If I turn off the preference "Show call tree in
TraceDebugger" am I
right to expect the Trace debugger behavior would be equivalent
to the
traditional one?
Yes and no. :-) First, the representation of the traced
program in
the TraceDebugger (stack vs tree) does not influence the
execution
semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable
to
locate certain contexts at certain points in time. That's why
these
contexts are skipped as you step through a program in the
TraceDebugger
with the context tree activated.
Second, code that is simulated inside the TraceDebugger is
(or
should) behave exactly as the same code being run in a normal
simulator
(like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including
we!) have
been working on fixing these bugs so that all code can behave
exactly
then same when being simulated. Still, there are some open known
(and
likely further unknown) issues (e.g., you cannot simulate a
simulator
which is executing a failed primitive: Context runSimulated:
[Context
runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:,
#terminateTo:, #findNextHandlerContextStarting) always fail when
the
context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio,
which
subclasses from Context (see SimulationContext) to make parts of
the
simulated code execution customizable. The VM, however, is not
prepared
to the existence of such subclass objects of Context and will
always
fail when these primitives are invoked on an object that is not
exactly
of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution
semantics
between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate
Simulator debug: [thisContext findNextHandlerContextStarting]
because
when I inspect thisContext in that debugger, it shows a subclass
of
Context; but when I do [thisContext
findNextHandlerContextStarting]
debugTrace, thisContext actually is an instance of Context
itself, so
how can the VM detect this? And you would be right, because when
you
*inspect* a context in the TraceDebugger, it is a Context
instance
indeed, but not when you actually *execute* it in the
TraceDebugger, as
you can see when you evaluate [thisContext class] debugTrace ...
The
explanation for that lies in TDBTrace>>#enableSimulatorDuring:,
but to
cut it short, we convert all (non-dead) Context instances from
the
TraceDebugger's tree to a subclass of SimulationContext
temporarily
during each step to achieve two things: First, to not confuse
observant
users like you with the existence of these subclasses (well,
maybe that
did not work too well), and second, to make it possible to
resume from
a trace debugger at any point, which will execute the process in
the
regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process
otherwise (you
can actually observe that when trying to proceed from Simulator
debug:
[thisContext findNextHandlerContextStarting]). (Fun fact: Not
all VMs
handle Context subinstances that carefully: SqueakJS will
seriously mix
up the context/object layout, while TruffleSqueak will terminate
as
soon as you instantiate (!) any subinstance of Context, so I'm
gladful
that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
> Example: do step through to the [^2] block and then step
through
again
> > [^2] ensure: [] > > Traditionally, you end up in the unwind block. > In the new Trace debugger you end up with the
#cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then
step
through again, I land in Context>>terminateTo:. Are you using
the
latest version of trunk and TraceDebugger? However, you
currently end
up in #cannotReturn: when stepping beyond the Processor
activeProcess
suspend in the bottom context of a process using the
TraceDebugger.
This is because other than the normal debugger, the
TraceDebugger does
not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
> Hi Christoph, > > sorry, a follow-up question :) > > If I turn off the preference "Show call tree in
TraceDebugger" am I
> right to expect the Trace debugger behavior would be
equivalent to
the
> traditional one? > > In other words: if I run the traditional debugger and the
Trace one
side
> by side, should they display analogous steps? > > In the other message I wrote about a different way to
simulate
primitive
> calls. > > However, I've noticed other irregularities so that's why I
started
> wondering maybe my assumption was wrong and the Trace
debugger is
> designed to present the simulation differently. Please
advise.
> > Example: do step through to the [^2] block and then step
through
again
> > [^2] ensure: [] > > Traditionally, you end up in the unwind block. > In the new Trace debugger you end up with the
#cannotReturn:
context as
> if the computation just ran until the end. > > Is this expected? (My guess is it isn't but can't figure
out why)
> > Thanks again, > Jaromir > > > > > > On 01-Jan-24 3:31:29 PM, "Jaromir Matas"
<mail(a)jaromir.net>
wrote:
> > >Hi Christoph, > > > >Is it ok that I ask questions about the new debugger? What
would
be the
> >best format for such a "Q&A" - here or perhaps within a
topic on
> >squeak-smalltalk/squeak-object-memory? I don't expect a
flood of
> >questions but to get a bit familiar with your debugger it
would
help
> >tremendously to be able to ask right away instead of
trudging
through
> >the code/help :) The code usually helps to understand
**how**
things
> >work, the mechanics, but rarely **why**, the intentions. > > > > > you can also turn off the preference "Show call tree in > >TraceDebugger" > > > >Thanks, that helps to familiarize myself with the new
functionalities
> >"step-by-step", and not be overwhelmed by all the might of
the
call
> >tree :) Being able to go back is already a hell of an
improvement!
Or
> >when you open an inspector on a context the context's
state gets
frozen
> >in time and won't change when you proceed debugging -
another
> >groundbreaking change! > > > >Question: > >In the traditional debugger, when you step into a
primitive, the
> >primitive gets executed and the simulation moves over the
primitive
> >call. The Trace debugger, however, starts executing the
fallback
code
> >of the primitive call - why is that? > > > >Screenshot after step into #terminateTo: > > > > > > > >Thanks again, > >Jaromir > > > >On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
> >wrote: > > > >>Hi Jaromir, > >> > >>thanks a lot for trying it out!! Your feedback means a
lot to me.
:-)
> >> > >>Indeed you hit a pretty unfortunate example. The
TraceDebugger is
not
> >>ready for all your clever investigations regarding
non-local
returns
> >>and unwinding. ;-) In fact, your example reveals another
limitation
> >>that I forgot to mention in the announcement, which
regards
programs
> >>with irregular context switches - e.g.,
generators/coroutines,
but
> >>also non-local returns through unwind contexts. This is
because
the
> >>TraceDebugger stores and displays all method invocations
in a
tree,
> >>but in the case of manual context switches, there is no
single
global
> >>tree - its structure would change over the execution
time, and
when
> >>selecting a method invocation, it is not even clear to
what
parent
> >>(sender) it would belong, as there might be multiple. The
current
> >>solution is to display the tree from the perspective of
the stack
at
> >>the viewed point in time (see also the '@ <timeIndex>' in
the
window
> >>title), so it looks corrupted while stepping through > >>Context>>#terminateTo: as the stack is being manipulated.
(You
would
> >>notice the same in a normal debugger if you turned off
the
optional
> >>primitive 196 in this method - for SimulationContexts
this method
> >>always uses the fallback code.) > >> > >>Nevertheless, I have pushed some changes that should
allow you to
step
> >>out of #terminateTo: again. (You can update the
TraceDebugger
from the
> >>window menu icon at the right top, like all of my tools.)
At some
> >>point there will no method be displayed, but you can just
step
further
> >>and eventually return back to the starting point. :-) If
you want
to,
> >>you can also turn off the preference "Show call tree in
TraceDebugger"
> >>to make the TraceDebugger look more like a normal
debugger, which
also
> >>solves the context switches issue. But in general -
unless you
are
> >>debugging unwinding stuff - I would not recommend that as
it
removes
> >>one important strength of the TraceDebugger. :-) > >> > >>But again, this is really not a prime example for the
TraceDebugger.
> >>Better use it to explore how the simulator works. :-) For
example, you
> >>could do the following: > >> > >>[ContextTest debug: #testBlockCannotReturn] debugTrace. > >> > >>And in that trace debugger, you could select the start
method,
press
> >>Cmd + f(ind), and type "return:from:" to investigate the
behavior
of
> >>your solution there again, etc. > >> > >>Thanks for your comments! This was a good chance for me
to sort
some
> >>things out! :-) > >> > >>Best, > >>Christoph > >> > >>--- > >>Sent from Squeak Inbox Talk > >>https://github.com/hpi-swa-lab/squeak-inbox-talk > >> > >>On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote: > >> > >> > Hi Christoph, > >> > > >> > This indeed sounds like a GREAT idea! I look forward
to seeing
your
> >>use > >> > cases to build the right intuition. > >> > > >> > In the meantime I've tried to debug/trace this example
I've
been
> >>working > >> > with lately: > >> > > >> > [^2] ensure: [] > >> > > >> > If I start the debugger, hit `trace it` and then `step
over`,
it
> >>stops > >> > at Context>>terminate and the view gets corrupted (the
initial
part
> >>of > >> > the trace is hidden and can't be made visible unless
clicking
on
> >>some of > >> > the pink lines - but not every line does it...) > >> > > >> > > >> > > >> > > >> > If I then continue stepping over it ends up with some
kind of
error:
> >> > > >> > > >> > Maybe this is just an unfortunate example... Or maybe
I'm just
doing
> >> > something wrong... > >> > > >> > At any rate - THANKS for your effort!! > >> > > >> > > >> > On 30-Dec-23 4:37:28 PM, > >>christoph.thiede(a)student.hpi.uni-potsdam.de > >> > wrote: > >> > > >> > >Thanks for the reply, Dave! I will try to post one or
two
concrete
> >>use > >> > >cases about the TraceDebugger in the next couple of
days, so
stay
> >> > >tuned. :-) > >> > > > >> > >Best, > >> > >Christoph > >> > > > >> > >--- > >> > >Sent from Squeak Inbox Talk > >> > >https://github.com/hpi-swa-lab/squeak-inbox-talk > >> > > > >> > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com
wrote:
> >> > > > >> > > > This sounds like really interesting work! I love
the idea
of
> >>being > >> > > > able to interactively go back in "oops, I‘ve
stepped too
far,
> >>let‘s > >> > > > start all over again" situations. It will probably
take
some
> >>time for > >> > > > me and others to wrap our heads around the things
you have
done,
> >>so > >> > > > don't be surprised if you get a delayed response
to this
> >>announcement > >> > > > :-) > >> > > > > >> > > > Congratulations! > >> > > > Dave > >> > > > > >> > > > > >> > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > >> > > > christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
> >> > > > > Hi all! > >> > > > > > >> > > > > I‘m very excited to announce a project today
that we
have been
> >> > > > > working on over the past two years: The
*TraceDebugger*
[1] is
> >>a > >> > >new > >> > > > > back-in-time/time-travel/omniscient debugging
tool for
Squeak
> >>that > >> > > > > allows you to record past method activations and
states
during
> >> > > > > execution and explore them later. > >> > > > > > >> > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger
> >> > > > > > >> > > > > Metacellonew > >> > > > > baseline:'TraceDebugger'; > >> > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
> >> > > > > /"repository: > >> > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
> >>Squeak > >> > > > > 6.0"/ > >> > > > > get; > >> > > > > load. > >> > > > > > >> > > > > **What can it do? (Features)** > >> > > > > > >> > > > > - *Record all method activations and historic
states:*
> >>Normally > >> > >step > >> > > > > through a program in the debugger while
automatically
> >>recording its > >> > > > > execution. > >> > > > > - *Replay execution of a traced program:*
Navigate
through all
> >> > > > > method invocations using the /*context tree*/ or
the
/*Step
> >> > >Back/Step > >> > > > > Forward*/ buttons (to avoid these "oops, I‘ve
stepped
too far,
> >> > > > > let‘s start all over again" situations). > >> > > > > - *Interact with historic states:*
Inspect/explore
snapshots
> >>of > >> > > > > objects or send them any message. > >> > > > > - *State-centric debugging using the ***/History > >>Explorer*/**:* > >> > > > > Gather, explore, and visualize all changes to an > >>object/expression > >> > > > > over the recorded time ("When did this > >> > >variable/collection/screenshot > >> > > > > change?"). > >> > > > > - *Additional navigation tools* for searching
and
filtering
> >>the > >> > > > > context tree. > >> > > > > - *Focus on interactivity:* No hours of
recording, no
GBs of
> >>mem > >> > > > > consumption - at least for common small to
medium
programs.
> >> > > > > - *UI resembles the classic Smalltalk debugger:*
You'll
find
> >>your > >> > > > > familiar stepping buttons, code browsing tools,
inspectors,
> >>and > >> > > > > shortcuts - plus more. > >> > > > > > >> > > > > The TraceDebugger is a general-purpose tool and
not tied
to
> >> > > > > particular domains. In the past months, we have
successfully
> >>used > >> > >it > >> > > > > to understand several bugs and interaction
patterns in
the
> >>Trunk > >> > > > > (Morphic layout/rendering, compiler/decompiler,
code
> >>simulation, > >> > > > > …). The tool is also self-supporting, so you can
debug a
> >> > > > > TraceDebugger from another TraceDebugger. :-) > >> > > > > > >> > > > > **What can‘t it do (yet)? (Limitations and
future
work)**
> >> > > > > > >> > > > > - *High performance:* While (sufficiently) fast
enough
for
> >>most > >> > > > > small to medium workloads, tracing very compute-
or
> >>mem-intensive > >> > > > > operations may require more time (ex.:
compiler/decompiler
> >> > > > > invocation: <1s, HTTPS request: <10s, tool
building:
<5m,
> >>complex > >> > > > > rendering: minutes up to hours). > >> > > > > - *Not a dataflow analyzer:* The TraceDebugger
does not
track
> >> > > > > dataflow events (e.g., argument passing) but
only state
> >>changes. > >> > > > > - *No tracing of external states/events* for
FFI/OSProcess or
> >> > >custom > >> > > > > VM modules. > >> > > > > - *No support for advanced language concepts*
such as
identity
> >> > > > > forwarding/write barriers. > >> > > > > > >> > > > > **How does it work? (Implementation)** > >> > > > > > >> > > > > In one sentence: To record message sends and
side
effects, we
> >> > > > > decorate the execution of certain bytecodes with
tracing
> >>extensions > >> > > > > by modifying the code simulation using
SimulationStudio
[2].
> >> > > > > > >> > > > > In one paragraph: The program is executed in a
specialized
> >>code > >> > > > > simulator that overrides instructions for
sending
messages
> >>(e.g., > >> > > > > send, superSend) and for performing side-effects
(e.g.,
> >> > >popIntoRcvr, > >> > > > > primitiveAtPut, push). All message sends are
recorded in
a
> >>tree and > >> > > > > all changed object slots are stored in a sparse
time-dependent
> >> > >memory > >> > > > > structure before they are overwritten. For
time-traveling, the
> >>tree > >> > > > > is traversed using a cursor. For accessing
historic
objects, a
> >> > >proxy > >> > > > > evaluates all messages sent to an object in
another
> >>specialized > >> > > > > simulator (retracing simulator) that emulates
historic
states
> >>for > >> > >the > >> > > > > requested point in time by forwarding read
primitives
(e.g.,
> >> > > > > pushRcvr, primitiveAt) to the recorded memory.
For
gathering
> >>state > >> > > > > changes in the History Explorer efficiently, the
query
is
> >>evaluated > >> > > > > in a range retracing simulator with
vectorization and
fork
> >> > >semantics. > >> > > > > > >> > > > > In academic terms: We have published two papers
about
the
> >> > > > > TraceDebugger that provide further details about
its
> >>implementation > >> > > > > and its applications for program exploration,
"Object-Centric
> >> > > > > Time-Travel Debugging: Exploring Traces of
Objects" [3]
and
> >> > > > > "Time-Awareness in Object Exploration Tools:
Toward In
Situ
> >> > > > > Omniscient Debugging" [4]. > >> > > > > > >> > > > > In Smalltalk: Just check out the code base and
explore
it by
> >> > > > > yourself! The class comments in TraceDebugger > >> > >code://TraceDebugger > >> > > > > and TDBCursor code://TDBCursor should provide
good
starting
> >> > >points. > >> > > > > > >> > > > > **How can I use it?** > >> > > > > > >> > > > > Please try it out and report feedback! The
TraceDebugger
> >>supports > >> > > > > the latest Squeak Trunk and Squeak 6.0. You can
either
> >>download a > >> > > > > prepared all-in-one bundle on GitHub: > >> > > > > > >> > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> >> > > > > > >> > > > > Or you can install it into your own image using
Metacello:
> >> > > > > > >> > > > > Metacellonew > >> > > > > baseline:'TraceDebugger'; > >> > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
> >> > > > > /"repository: > >> > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
> >>Squeak > >> > > > > 6.0"/ > >> > > > > get; > >> > > > > load. > >> > > > > > >> > > > > To get started, just open a normal debugger
(e.g., by
> >>selecting an > >> > > > > expression and pressing Cmd+Shift+D to debug it)
and
then
> >>press the > >> > > > > "Trace It" button on the right. There‘s also
some pretty
> >>detailed > >> > > > > documentation in the Help Browser <code://
TraceDebugger
> >>showHelp> > >> > > > > that covers everything you should know. > >> > > > > > >> > > > > My goal is to improve convenience and provide a
useful
tool
> >>for the > >> > > > > community, so I‘m very excited to hear your
impressions,
> >>ideas, and > >> > > > > thoughts. Here, on GitHub, or in a private
message.
Let‘s have
> >>a > >> > > > > great discussion! :-) > >> > > > > > >> > > > > Best, > >> > > > > > >> > > > > Christoph (and Marcel) > >> > > > > > >> > > > > PS: Props to Eliot who brought up the original
idea of
> >>"subclassing > >> > > > > from Context" for other reasons four years ago.
[5]
> >> > > > > > >> > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
> >> > > > > [2]
https://github.com/LinqLover/SimulationStudio
> >> > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> >>2023. > >> > > > > Object-Centric Time-Travel Debugging: Exploring
Traces
of
> >>Objects. > >> > > > > https://doi.org/10.1145/3594671.3594678 In
/Companion
> >>Proceedings > >> > > > > of the 7th International Conference on the Art,
Science,
and
> >> > > > > Engineering of Programming/ (/<Programming>'23
Companion/),
> >>March > >> > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY,
USA, 7
pages.
> >>DOI: > >> > > > > 10.1145/3594671.3594678 > >>https://doi.org/10.1145/3594671.3594678. > >> > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
> >> > > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> >>2023. > >> > > > > Time-Awareness in Object Exploration Tools:
Toward In
Situ
> >> > >Omniscient > >> > > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> >> > > > > /Proceedings of the 2023 ACM SIGPLAN
International
Symposium
> >>on New > >> > > > > Ideas, New Paradigms, and Reflections on
Programming and
> >>Software/ > >> > > > > (/Onward! '23/), October 25–27, 2023, Cascais,
Portugal.
ACM,
> >>New > >> > > > > York, NY, USA, 14 pages. DOI:
10.1145/3622758.3622892
> >> > > > > https://doi.org/10.1145/3622758.3622892. PDF: > >> > > > >
https://dl.acm.org/doi/pdf/10.1145/3622758.3622892
> >> > > > > [5] > >> > > > > > >> > > >>
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> >> > > > > > >> > > > > --- > >> > > > > /Sent from//Squeak Inbox Talk > >> > > > >
Hi Christoph,
Thanks a lot for your explanations - very helpful!
a few more notes inlined:
On 02-Jan-24 7:53:31 PM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an inspector and sometimes I desperately want to compare past and present states. It almost sounds like a preference :) Btw. I usually use the Explorer instead of Inspector but what annoys me is the Explorer (unlike the Inspector) doesn't update the states automatically: I have to refresh manually (switch to Inspector and back or collapse the hierarchy and open again - is there possibly something to make the Explorer update automatically? Thanks!)
But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I
right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: ContextrunSimulated:[ContextrunSimulated:[#()tryPrimitive:60withArgs:#(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
Understood, thanks. I have noticed, however, that your stepping methods like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come? Is there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
However, now you might say: This makes sense when I evaluate Simulatordebug:[thisContextfindNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContextfindNextHandlerContextStarting]debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContextclass]debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulatordebug:[thisContextfindNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Will study, thanks!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context
as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the ProcessoractiveProcesssuspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-01T19:25:04+00:00, mail@jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
In other words: if I run the traditional debugger and the Trace one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context
as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net> wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would be
the
best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would help tremendously to be able to ask right away instead of trudging
through
the code/help :) The code usually helps to understand **how** things work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new
functionalities
"step-by-step", and not be overwhelmed by all the might of the call tree :) Being able to go back is already a hell of an improvement!
Or
when you open an inspector on a context the context's state gets
frozen
in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the primitive call. The Trace debugger, however, starts executing the fallback
code
of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me.
:-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is
not
ready for all your clever investigations regarding non-local
returns
and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards
programs
with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single
global
tree - its structure would change over the execution time, and when selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack
at
the viewed point in time (see also the '@ <timeIndex>' in the
window
title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You
would
notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to
step
out of #terminateTo: again. (You can update the TraceDebugger from
the
window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step
further
and eventually return back to the starting point. :-) If you want
to,
you can also turn off the preference "Show call tree in
TraceDebugger"
to make the TraceDebugger look more like a normal debugger, which
also
solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it
removes
one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the
TraceDebugger.
Better use it to explore how the simulator works. :-) For example,
you
could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method,
press
Cmd + f(ind), and type "return:from:" to investigate the behavior
of
your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort
some
things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing
your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it
stops
at Context>>terminate and the view gets corrupted (the initial
part
of
the trace is hidden and can't be made visible unless clicking on
some of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of
error:
Maybe this is just an unfortunate example... Or maybe I'm just
doing
something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Thanks for the reply, Dave! I will try to post one or two
concrete
use
cases about the TraceDebugger in the next couple of days, so
stay
tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote:
> This sounds like really interesting work! I love the idea of
being
> able to interactively go back in "oops, I‘ve stepped too
far,
let‘s
> start all over again" situations. It will probably take some
time for
> me and others to wrap our heads around the things you have
done,
so
> don't be surprised if you get a delayed response to this
announcement
> :-) > > Congratulations! > Dave > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > Hi all! > > > > I‘m very excited to announce a project today that we have
been
> > working on over the past two years: The *TraceDebugger*
[1] is
a
new > > back-in-time/time-travel/omniscient debugging tool for
Squeak
that
> > allows you to record past method activations and states
during
> > execution and explore them later. > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > Metacellonew > > baseline:'TraceDebugger'; > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > /"repository: > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > 6.0"/ > > get; > > load. > > > > **What can it do? (Features)** > > > > - *Record all method activations and historic states:*
Normally
step > > through a program in the debugger while automatically
recording its
> > execution. > > - *Replay execution of a traced program:* Navigate through
all
> > method invocations using the /*context tree*/ or the
/*Step
Back/Step > > Forward*/ buttons (to avoid these "oops, I‘ve stepped too
far,
> > let‘s start all over again" situations). > > - *Interact with historic states:* Inspect/explore
snapshots
of
> > objects or send them any message. > > - *State-centric debugging using the ***/History
Explorer*/**:*
> > Gather, explore, and visualize all changes to an
object/expression
> > over the recorded time ("When did this variable/collection/screenshot > > change?"). > > - *Additional navigation tools* for searching and
filtering
the
> > context tree. > > - *Focus on interactivity:* No hours of recording, no GBs
of
mem
> > consumption - at least for common small to medium
programs.
> > - *UI resembles the classic Smalltalk debugger:* You'll
find
your
> > familiar stepping buttons, code browsing tools,
inspectors,
and
> > shortcuts - plus more. > > > > The TraceDebugger is a general-purpose tool and not tied
to
> > particular domains. In the past months, we have
successfully
used
it > > to understand several bugs and interaction patterns in the
Trunk
> > (Morphic layout/rendering, compiler/decompiler, code
simulation,
> > …). The tool is also self-supporting, so you can debug a > > TraceDebugger from another TraceDebugger. :-) > > > > **What can‘t it do (yet)? (Limitations and future work)** > > > > - *High performance:* While (sufficiently) fast enough for
most
> > small to medium workloads, tracing very compute- or
mem-intensive
> > operations may require more time (ex.: compiler/decompiler > > invocation: <1s, HTTPS request: <10s, tool building: <5m,
complex
> > rendering: minutes up to hours). > > - *Not a dataflow analyzer:* The TraceDebugger does not
track
> > dataflow events (e.g., argument passing) but only state
changes.
> > - *No tracing of external states/events* for FFI/OSProcess
or
custom > > VM modules. > > - *No support for advanced language concepts* such as
identity
> > forwarding/write barriers. > > > > **How does it work? (Implementation)** > > > > In one sentence: To record message sends and side effects,
we
> > decorate the execution of certain bytecodes with tracing
extensions
> > by modifying the code simulation using SimulationStudio
[2].
> > > > In one paragraph: The program is executed in a specialized
code
> > simulator that overrides instructions for sending messages
(e.g.,
> > send, superSend) and for performing side-effects (e.g., popIntoRcvr, > > primitiveAtPut, push). All message sends are recorded in a
tree and
> > all changed object slots are stored in a sparse
time-dependent
memory > > structure before they are overwritten. For time-traveling,
the
tree
> > is traversed using a cursor. For accessing historic
objects, a
proxy > > evaluates all messages sent to an object in another
specialized
> > simulator (retracing simulator) that emulates historic
states
for
the > > requested point in time by forwarding read primitives
(e.g.,
> > pushRcvr, primitiveAt) to the recorded memory. For
gathering
state
> > changes in the History Explorer efficiently, the query is
evaluated
> > in a range retracing simulator with vectorization and fork semantics. > > > > In academic terms: We have published two papers about the > > TraceDebugger that provide further details about its
implementation
> > and its applications for program exploration,
"Object-Centric
> > Time-Travel Debugging: Exploring Traces of Objects" [3]
and
> > "Time-Awareness in Object Exploration Tools: Toward In
Situ
> > Omniscient Debugging" [4]. > > > > In Smalltalk: Just check out the code base and explore it
by
> > yourself! The class comments in TraceDebugger code://TraceDebugger > > and TDBCursor code://TDBCursor should provide good
starting
points. > > > > **How can I use it?** > > > > Please try it out and report feedback! The TraceDebugger
supports
> > the latest Squeak Trunk and Squeak 6.0. You can either
download a
> > prepared all-in-one bundle on GitHub: > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> > > > Or you can install it into your own image using Metacello: > > > > Metacellonew > > baseline:'TraceDebugger'; > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > /"repository: > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > 6.0"/ > > get; > > load. > > > > To get started, just open a normal debugger (e.g., by
selecting an
> > expression and pressing Cmd+Shift+D to debug it) and then
press the
> > "Trace It" button on the right. There‘s also some pretty
detailed
> > documentation in the Help Browser <code:// TraceDebugger
showHelp>
> > that covers everything you should know. > > > > My goal is to improve convenience and provide a useful
tool
for the
> > community, so I‘m very excited to hear your impressions,
ideas, and
> > thoughts. Here, on GitHub, or in a private message. Let‘s
have
a
> > great discussion! :-) > > > > Best, > > > > Christoph (and Marcel) > > > > PS: Props to Eliot who brought up the original idea of
"subclassing
> > from Context" for other reasons four years ago. [5] > > > > [1] https://github.com/hpi-swa-lab/squeak-tracedebugger > > [2] https://github.com/LinqLover/SimulationStudio > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > Object-Centric Time-Travel Debugging: Exploring Traces of
Objects.
> > https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
> > of the 7th International Conference on the Art, Science,
and
> > Engineering of Programming/ (/<Programming>'23
Companion/),
March
> > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
DOI:
> > 10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
> > PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
on New
> > Ideas, New Paradigms, and Reflections on Programming and
Software/
> > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal.
ACM,
New
> > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > https://doi.org/10.1145/3622758.3622892. PDF: > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > [5] > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > --- > > /Sent from//Squeak Inbox Talk > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Jaromir,
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an inspector and sometimes I desperately want to compare past and present states. It almost sounds like a preference :)
Hm, I need to think about this ... in the meantime, you can yellow-click an object in a TraceDebugger's/snapshot inspector > basic inspect then yellow-click object > inspect to get its current version. Not very convenient. PRs to TraceDebugger are welcome. :-)
Btw. I usually use the Explorer instead of Inspector but what annoys me is the Explorer (unlike the Inspector) doesn't update the states automatically: I have to refresh manually (switch to Inspector and back or collapse the hierarchy and open again - is there possibly something to make the Explorer update automatically? Thanks!)
Now you are talking about Squeak in general, right? :) You can yellow-click a node in an explorer and choose "monitor changes". Maybe this should even be the default? But as you say, snapshots can be helpful in certain situations ... fascinating how a limitation can evolve as a feature ...
(By the way, there is also a preference called "Replace inspectors with explorers in TraceDebugger". But it does not really look nice, that's why its labeled as experimental.)
Understood, thanks. I have noticed, however, that your stepping methods like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come? Is there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
I would say it's a mixture of both - refactoring and a different model. I put some effort in separating model (TDBCursor) and view (TraceDebugger) from each other better than the Debugger does (so TDBCursor is also usable by itself as a scriptable time-travel debugger). The stepping logic in the normal Debugger is also a bit scattered in Debugger, Process, and Context. Some things are not required in the TraceDebugger, such as the need for the ingenious #runUntilErrorOrReturnFrom: because all contexts during tracing must be simulated. Some things also cannot work like in the normal Debugger due to the possibility of stepping backwards, such as the updates to the window label in case of an unhandled error: Normal Debugger uses a Notification for this (see #handleLabelUpdatesIn:whenExecuting:), which only works one-way, whereas the TraceDebugger uses a more functional approach to detect whether the exception handler is still on the stack (see TraceDebugger>>#labelString). Oops, this seems to be broken in the trunk: the "ZeroDivide:" should actually disappear again once after some steps ...
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2024-01-04T00:03:13+00:00, mail@jaromir.net wrote:
Hi Christoph,
Thanks a lot for your explanations - very helpful!
a few more notes inlined:
On 02-Jan-24 7:53:31 PM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an inspector and sometimes I desperately want to compare past and present states. It almost sounds like a preference :) Btw. I usually use the Explorer instead of Inspector but what annoys me is the Explorer (unlike the Inspector) doesn't update the states automatically: I have to refresh manually (switch to Inspector and back or collapse the hierarchy and open again - is there possibly something to make the Explorer update automatically? Thanks!)
But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I
right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: ContextrunSimulated:[ContextrunSimulated:[#()tryPrimitive:60withArgs:#(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
Understood, thanks. I have noticed, however, that your stepping methods like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come? Is there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
However, now you might say: This makes sense when I evaluate Simulatordebug:[thisContextfindNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContextfindNextHandlerContextStarting]debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContextclass]debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulatordebug:[thisContextfindNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Will study, thanks!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context
as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the ProcessoractiveProcesssuspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
In other words: if I run the traditional debugger and the Trace one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context
as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net> wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would be
the
best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would help tremendously to be able to ask right away instead of trudging
through
the code/help :) The code usually helps to understand **how** things work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new
functionalities
"step-by-step", and not be overwhelmed by all the might of the call tree :) Being able to go back is already a hell of an improvement!
Or
when you open an inspector on a context the context's state gets
frozen
in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the primitive call. The Trace debugger, however, starts executing the fallback
code
of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me.
:-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is
not
ready for all your clever investigations regarding non-local
returns
and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards
programs
with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single
global
tree - its structure would change over the execution time, and when selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack
at
the viewed point in time (see also the '@ <timeIndex>' in the
window
title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You
would
notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to
step
out of #terminateTo: again. (You can update the TraceDebugger from
the
window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step
further
and eventually return back to the starting point. :-) If you want
to,
you can also turn off the preference "Show call tree in
TraceDebugger"
to make the TraceDebugger look more like a normal debugger, which
also
solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it
removes
one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the
TraceDebugger.
Better use it to explore how the simulator works. :-) For example,
you
could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method,
press
Cmd + f(ind), and type "return:from:" to investigate the behavior
of
your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort
some
things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing
your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it
stops
at Context>>terminate and the view gets corrupted (the initial
part
of
the trace is hidden and can't be made visible unless clicking on
some of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of
error:
Maybe this is just an unfortunate example... Or maybe I'm just
doing
something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
>Thanks for the reply, Dave! I will try to post one or two
concrete
use
>cases about the TraceDebugger in the next couple of days, so
stay
>tuned. :-) > >Best, >Christoph > >--- >Sent from Squeak Inbox Talk >https://github.com/hpi-swa-lab/squeak-inbox-talk > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote: > > > This sounds like really interesting work! I love the idea of
being
> > able to interactively go back in "oops, I‘ve stepped too
far,
let‘s
> > start all over again" situations. It will probably take some
time for
> > me and others to wrap our heads around the things you have
done,
so
> > don't be surprised if you get a delayed response to this
announcement
> > :-) > > > > Congratulations! > > Dave > > > > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > > Hi all! > > > > > > I‘m very excited to announce a project today that we have
been
> > > working on over the past two years: The *TraceDebugger*
[1] is
a
>new > > > back-in-time/time-travel/omniscient debugging tool for
Squeak
that
> > > allows you to record past method activations and states
during
> > > execution and explore them later. > > > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > > > Metacellonew > > > baseline:'TraceDebugger'; > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > /"repository: > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > > 6.0"/ > > > get; > > > load. > > > > > > **What can it do? (Features)** > > > > > > - *Record all method activations and historic states:*
Normally
>step > > > through a program in the debugger while automatically
recording its
> > > execution. > > > - *Replay execution of a traced program:* Navigate through
all
> > > method invocations using the /*context tree*/ or the
/*Step
>Back/Step > > > Forward*/ buttons (to avoid these "oops, I‘ve stepped too
far,
> > > let‘s start all over again" situations). > > > - *Interact with historic states:* Inspect/explore
snapshots
of
> > > objects or send them any message. > > > - *State-centric debugging using the ***/History
Explorer*/**:*
> > > Gather, explore, and visualize all changes to an
object/expression
> > > over the recorded time ("When did this >variable/collection/screenshot > > > change?"). > > > - *Additional navigation tools* for searching and
filtering
the
> > > context tree. > > > - *Focus on interactivity:* No hours of recording, no GBs
of
mem
> > > consumption - at least for common small to medium
programs.
> > > - *UI resembles the classic Smalltalk debugger:* You'll
find
your
> > > familiar stepping buttons, code browsing tools,
inspectors,
and
> > > shortcuts - plus more. > > > > > > The TraceDebugger is a general-purpose tool and not tied
to
> > > particular domains. In the past months, we have
successfully
used
>it > > > to understand several bugs and interaction patterns in the
Trunk
> > > (Morphic layout/rendering, compiler/decompiler, code
simulation,
> > > …). The tool is also self-supporting, so you can debug a > > > TraceDebugger from another TraceDebugger. :-) > > > > > > **What can‘t it do (yet)? (Limitations and future work)** > > > > > > - *High performance:* While (sufficiently) fast enough for
most
> > > small to medium workloads, tracing very compute- or
mem-intensive
> > > operations may require more time (ex.: compiler/decompiler > > > invocation: <1s, HTTPS request: <10s, tool building: <5m,
complex
> > > rendering: minutes up to hours). > > > - *Not a dataflow analyzer:* The TraceDebugger does not
track
> > > dataflow events (e.g., argument passing) but only state
changes.
> > > - *No tracing of external states/events* for FFI/OSProcess
or
>custom > > > VM modules. > > > - *No support for advanced language concepts* such as
identity
> > > forwarding/write barriers. > > > > > > **How does it work? (Implementation)** > > > > > > In one sentence: To record message sends and side effects,
we
> > > decorate the execution of certain bytecodes with tracing
extensions
> > > by modifying the code simulation using SimulationStudio
[2].
> > > > > > In one paragraph: The program is executed in a specialized
code
> > > simulator that overrides instructions for sending messages
(e.g.,
> > > send, superSend) and for performing side-effects (e.g., >popIntoRcvr, > > > primitiveAtPut, push). All message sends are recorded in a
tree and
> > > all changed object slots are stored in a sparse
time-dependent
>memory > > > structure before they are overwritten. For time-traveling,
the
tree
> > > is traversed using a cursor. For accessing historic
objects, a
>proxy > > > evaluates all messages sent to an object in another
specialized
> > > simulator (retracing simulator) that emulates historic
states
for
>the > > > requested point in time by forwarding read primitives
(e.g.,
> > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
state
> > > changes in the History Explorer efficiently, the query is
evaluated
> > > in a range retracing simulator with vectorization and fork >semantics. > > > > > > In academic terms: We have published two papers about the > > > TraceDebugger that provide further details about its
implementation
> > > and its applications for program exploration,
"Object-Centric
> > > Time-Travel Debugging: Exploring Traces of Objects" [3]
and
> > > "Time-Awareness in Object Exploration Tools: Toward In
Situ
> > > Omniscient Debugging" [4]. > > > > > > In Smalltalk: Just check out the code base and explore it
by
> > > yourself! The class comments in TraceDebugger >code://TraceDebugger > > > and TDBCursor code://TDBCursor should provide good
starting
>points. > > > > > > **How can I use it?** > > > > > > Please try it out and report feedback! The TraceDebugger
supports
> > > the latest Squeak Trunk and Squeak 6.0. You can either
download a
> > > prepared all-in-one bundle on GitHub: > > > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> > > > > > Or you can install it into your own image using Metacello: > > > > > > Metacellonew > > > baseline:'TraceDebugger'; > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > /"repository: > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > > 6.0"/ > > > get; > > > load. > > > > > > To get started, just open a normal debugger (e.g., by
selecting an
> > > expression and pressing Cmd+Shift+D to debug it) and then
press the
> > > "Trace It" button on the right. There‘s also some pretty
detailed
> > > documentation in the Help Browser <code:// TraceDebugger
showHelp>
> > > that covers everything you should know. > > > > > > My goal is to improve convenience and provide a useful
tool
for the
> > > community, so I‘m very excited to hear your impressions,
ideas, and
> > > thoughts. Here, on GitHub, or in a private message. Let‘s
have
a
> > > great discussion! :-) > > > > > > Best, > > > > > > Christoph (and Marcel) > > > > > > PS: Props to Eliot who brought up the original idea of
"subclassing
> > > from Context" for other reasons four years ago. [5] > > > > > > [1] https://github.com/hpi-swa-lab/squeak-tracedebugger > > > [2] https://github.com/LinqLover/SimulationStudio > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > Object-Centric Time-Travel Debugging: Exploring Traces of
Objects.
> > > https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
> > > of the 7th International Conference on the Art, Science,
and
> > > Engineering of Programming/ (/<Programming>'23
Companion/),
March
> > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
DOI:
> > > 10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
> > > PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > Time-Awareness in Object Exploration Tools: Toward In Situ >Omniscient > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
on New
> > > Ideas, New Paradigms, and Reflections on Programming and
Software/
> > > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal.
ACM,
New
> > > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > > https://doi.org/10.1145/3622758.3622892. PDF: > > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > > [5] > > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > > > --- > > > /Sent from//Squeak Inbox Talk > > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Christoph, thanks again for your help!
On 04-Jan-24 1:59:00 AM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an
inspector and sometimes I desperately want to compare past and present states. It almost sounds like a preference :)
Hm, I need to think about this ... in the meantime, you can yellow-click an object in a TraceDebugger's/snapshot inspector > basic inspect then yellow-click object > inspect to get its current version. Not very convenient. PRs to TraceDebugger are welcome. :-)
Btw. I usually use the Explorer instead of Inspector but what annoys
me is the Explorer (unlike the Inspector) doesn't update the states automatically: I have to refresh manually (switch to Inspector and back or collapse the hierarchy and open again - is there possibly something to make the Explorer update automatically? Thanks!)
Now you are talking about Squeak in general, right? :) You can yellow-click a node in an explorer and choose "monitor changes". Maybe this should even be the default?
OMG, never noticed... and wasted so much time :D Yes, I'd welcome this behavior as the default (it bit me a few times in the begining before I noticed it wouldn't update in real time) - that would make it consistent with the Inspector's behavior (display 'all inst vars' kind of helps to monitor all fieds but looks a bit untidy).
But as you say, snapshots can be helpful in certain situations ... fascinating how a limitation can evolve as a feature ...
(By the way, there is also a preference called "Replace inspectors with explorers in TraceDebugger". But it does not really look nice, that's why its labeled as experimental.)
Cool! For this to look more reasonably you could hide the "value" areas that display the Inspector's values... (screenshot). The debugging is so much more dynamic this way :)
Understood, thanks. I have noticed, however, that your stepping
methods like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come? Is there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
I would say it's a mixture of both - refactoring and a different model. I put some effort in separating model (TDBCursor) and view (TraceDebugger) from each other better than the Debugger does (so TDBCursor is also usable by itself as a scriptable time-travel debugger). The stepping logic in the normal Debugger is also a bit scattered in Debugger, Process, and Context. Some things are not required in the TraceDebugger, such as the need for the ingenious #runUntilErrorOrReturnFrom: because all contexts during tracing must be simulated.
Ahh, true... is this the source of the performance limitation? I tried to debug some stepOver execution in the Trace debugger and sometimes a step takes a minute or two.
Some things also cannot work like in the normal Debugger due to the possibility of stepping backwards, such as the updates to the window label in case of an unhandled error: Normal Debugger uses a Notification for this (see #handleLabelUpdatesIn:whenExecuting:), which only works one-way, whereas the TraceDebugger uses a more functional approach to detect whether the exception handler is still on the stack (see TraceDebugger>>#labelString). Oops, this seems to be broken in the trunk: the "ZeroDivide:" should actually disappear again once after some steps ...
Thanks again!
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-04T00:03:13+00:00, mail@jaromir.net wrote:
Hi Christoph,
Thanks a lot for your explanations - very helpful!
a few more notes inlined:
On 02-Jan-24 7:53:31 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev
is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded Collections-ct.1061 to the inbox which would fix it. However, it is
a
bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so
maybe
this needs further discussion.
Or when you open an inspector on a context the context's state
gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows
as
I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might
be
confusing as well because there is no clear visual connection ...
It's
an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an
inspector
and sometimes I desperately want to compare past and present states.
It
almost sounds like a preference :) Btw. I usually use the Explorer instead of Inspector but what annoys me is the Explorer (unlike the Inspector) doesn't update the states automatically: I have to refresh manually (switch to Inspector and back or collapse the hierarchy and open again - is there possibly something to make the Explorer update automatically? Thanks!)
But I'm glad they work well for you. :-) If you have any better
ideas,
let me know!
Regarding your questions about the behavior of code when being run
in
the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am
I
right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
Yes and no. :-) First, the representation of the traced program in
the
TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the
TraceDebugger
with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should)
behave exactly as the same code being run in a normal simulator
(like
when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have been working on fixing these bugs so that all code can behave
exactly
then same when being simulated. Still, there are some open known
(and
likely further unknown) issues (e.g., you cannot simulate a
simulator
which is executing a failed primitive:
ContextrunSimulated:[ContextrunSimulated:[#()tryPrimitive:60withArgs:#(0)]]),
so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not
prepared
to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not
exactly
of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
Understood, thanks. I have noticed, however, that your stepping
methods
like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come? Is there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
However, now you might say: This makes sense when I evaluate Simulatordebug:[thisContextfindNextHandlerContextStarting] because
when
I inspect thisContext in that debugger, it shows a subclass of
Context;
but when I do [thisContextfindNextHandlerContextStarting]debugTrace, thisContext actually is an instance of Context itself, so how can
the
VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but
not
when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContextclass]debugTrace ... The explanation
for
that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short,
we
convert all (non-dead) Context instances from the TraceDebugger's
tree
to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you
with
the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace
debugger
at any point, which will execute the process in the regular VM; and
as
noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulatordebug:[thisContextfindNextHandlerContextStarting]). (Fun
fact:
Not all VMs handle Context subinstances that carefully: SqueakJS
will
seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context,
so
I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Will study, thanks!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context
as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through
again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the
ProcessoractiveProcesssuspend
in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not
yet
honor the suspended/terminated state of the interrupted process.
Maybe
it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am
I
right to expect the Trace debugger behavior would be equivalent
to
the
traditional one?
In other words: if I run the traditional debugger and the Trace
one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I
started
wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context
as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would
be
the
best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would
help
tremendously to be able to ask right away instead of trudging
through
the code/help :) The code usually helps to understand **how**
things
work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new
functionalities
"step-by-step", and not be overwhelmed by all the might of the
call
tree :) Being able to go back is already a hell of an
improvement!
Or
when you open an inspector on a context the context's state gets
frozen
in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the
primitive
call. The Trace debugger, however, starts executing the fallback
code
of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to
me.
:-)
Indeed you hit a pretty unfortunate example. The TraceDebugger
is
not
ready for all your clever investigations regarding non-local
returns
and unwinding. ;-) In fact, your example reveals another
limitation
that I forgot to mention in the announcement, which regards
programs
with irregular context switches - e.g., generators/coroutines,
but
also non-local returns through unwind contexts. This is because
the
TraceDebugger stores and displays all method invocations in a
tree,
but in the case of manual context switches, there is no single
global
tree - its structure would change over the execution time, and
when
selecting a method invocation, it is not even clear to what
parent
(sender) it would belong, as there might be multiple. The
current
solution is to display the tree from the perspective of the
stack
at
the viewed point in time (see also the '@ <timeIndex>' in the
window
title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You
would
notice the same in a normal debugger if you turned off the
optional
primitive 196 in this method - for SimulationContexts this
method
always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you
to
step
out of #terminateTo: again. (You can update the TraceDebugger
from
the
window menu icon at the right top, like all of my tools.) At
some
point there will no method be displayed, but you can just step
further
and eventually return back to the starting point. :-) If you
want
to,
you can also turn off the preference "Show call tree in
TraceDebugger"
to make the TraceDebugger look more like a normal debugger,
which
also
solves the context switches issue. But in general - unless you
are
debugging unwinding stuff - I would not recommend that as it
removes
one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the
TraceDebugger.
Better use it to explore how the simulator works. :-) For
example,
you
could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method,
press
Cmd + f(ind), and type "return:from:" to investigate the
behavior
of
your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort
some
things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
> Hi Christoph, > > This indeed sounds like a GREAT idea! I look forward to
seeing
your
use > cases to build the right intuition. > > In the meantime I've tried to debug/trace this example I've
been
working > with lately: > > [^2] ensure: [] > > If I start the debugger, hit `trace it` and then `step
over`, it
stops > at Context>>terminate and the view gets corrupted (the
initial
part
of > the trace is hidden and can't be made visible unless
clicking on
some of > the pink lines - but not every line does it...) > > > > > If I then continue stepping over it ends up with some kind
of
error:
> > > Maybe this is just an unfortunate example... Or maybe I'm
just
doing
> something wrong... > > At any rate - THANKS for your effort!! > > > On 30-Dec-23 4:37:28 PM, christoph.thiede(a)student.hpi.uni-potsdam.de > wrote: > > >Thanks for the reply, Dave! I will try to post one or two
concrete
use > >cases about the TraceDebugger in the next couple of days,
so
stay
> >tuned. :-) > > > >Best, > >Christoph > > > >--- > >Sent from Squeak Inbox Talk > >https://github.com/hpi-swa-lab/squeak-inbox-talk > > > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote: > > > > > This sounds like really interesting work! I love the
idea of
being > > > able to interactively go back in "oops, I‘ve stepped too
far,
let‘s > > > start all over again" situations. It will probably take
some
time for > > > me and others to wrap our heads around the things you
have
done,
so > > > don't be surprised if you get a delayed response to this announcement > > > :-) > > > > > > Congratulations! > > > Dave > > > > > > > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > > > Hi all! > > > > > > > > I‘m very excited to announce a project today that we
have
been
> > > > working on over the past two years: The
*TraceDebugger*
[1] is
a > >new > > > > back-in-time/time-travel/omniscient debugging tool for
Squeak
that > > > > allows you to record past method activations and
states
during
> > > > execution and explore them later. > > > > > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > > > > > Metacellonew > > > > baseline:'TraceDebugger'; > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
> > > > /"repository: > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak > > > > 6.0"/ > > > > get; > > > > load. > > > > > > > > **What can it do? (Features)** > > > > > > > > - *Record all method activations and historic states:* Normally > >step > > > > through a program in the debugger while automatically recording its > > > > execution. > > > > - *Replay execution of a traced program:* Navigate
through
all
> > > > method invocations using the /*context tree*/ or the
/*Step
> >Back/Step > > > > Forward*/ buttons (to avoid these "oops, I‘ve stepped
too
far,
> > > > let‘s start all over again" situations). > > > > - *Interact with historic states:* Inspect/explore
snapshots
of > > > > objects or send them any message. > > > > - *State-centric debugging using the ***/History Explorer*/**:* > > > > Gather, explore, and visualize all changes to an object/expression > > > > over the recorded time ("When did this > >variable/collection/screenshot > > > > change?"). > > > > - *Additional navigation tools* for searching and
filtering
the > > > > context tree. > > > > - *Focus on interactivity:* No hours of recording, no
GBs
of
mem > > > > consumption - at least for common small to medium
programs.
> > > > - *UI resembles the classic Smalltalk debugger:*
You'll
find
your > > > > familiar stepping buttons, code browsing tools,
inspectors,
and > > > > shortcuts - plus more. > > > > > > > > The TraceDebugger is a general-purpose tool and not
tied
to
> > > > particular domains. In the past months, we have
successfully
used > >it > > > > to understand several bugs and interaction patterns in
the
Trunk > > > > (Morphic layout/rendering, compiler/decompiler, code simulation, > > > > …). The tool is also self-supporting, so you can debug
a
> > > > TraceDebugger from another TraceDebugger. :-) > > > > > > > > **What can‘t it do (yet)? (Limitations and future
work)**
> > > > > > > > - *High performance:* While (sufficiently) fast enough
for
most > > > > small to medium workloads, tracing very compute- or mem-intensive > > > > operations may require more time (ex.:
compiler/decompiler
> > > > invocation: <1s, HTTPS request: <10s, tool building:
<5m,
complex > > > > rendering: minutes up to hours). > > > > - *Not a dataflow analyzer:* The TraceDebugger does
not
track
> > > > dataflow events (e.g., argument passing) but only
state
changes. > > > > - *No tracing of external states/events* for
FFI/OSProcess
or
> >custom > > > > VM modules. > > > > - *No support for advanced language concepts* such as
identity
> > > > forwarding/write barriers. > > > > > > > > **How does it work? (Implementation)** > > > > > > > > In one sentence: To record message sends and side
effects,
we
> > > > decorate the execution of certain bytecodes with
tracing
extensions > > > > by modifying the code simulation using
SimulationStudio
[2].
> > > > > > > > In one paragraph: The program is executed in a
specialized
code > > > > simulator that overrides instructions for sending
messages
(e.g., > > > > send, superSend) and for performing side-effects
(e.g.,
> >popIntoRcvr, > > > > primitiveAtPut, push). All message sends are recorded
in a
tree and > > > > all changed object slots are stored in a sparse
time-dependent
> >memory > > > > structure before they are overwritten. For
time-traveling,
the
tree > > > > is traversed using a cursor. For accessing historic
objects, a
> >proxy > > > > evaluates all messages sent to an object in another specialized > > > > simulator (retracing simulator) that emulates historic
states
for > >the > > > > requested point in time by forwarding read primitives
(e.g.,
> > > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
state > > > > changes in the History Explorer efficiently, the query
is
evaluated > > > > in a range retracing simulator with vectorization and
fork
> >semantics. > > > > > > > > In academic terms: We have published two papers about
the
> > > > TraceDebugger that provide further details about its implementation > > > > and its applications for program exploration,
"Object-Centric
> > > > Time-Travel Debugging: Exploring Traces of Objects"
[3]
and
> > > > "Time-Awareness in Object Exploration Tools: Toward In
Situ
> > > > Omniscient Debugging" [4]. > > > > > > > > In Smalltalk: Just check out the code base and explore
it
by
> > > > yourself! The class comments in TraceDebugger > >code://TraceDebugger > > > > and TDBCursor code://TDBCursor should provide good
starting
> >points. > > > > > > > > **How can I use it?** > > > > > > > > Please try it out and report feedback! The
TraceDebugger
supports > > > > the latest Squeak Trunk and Squeak 6.0. You can either download a > > > > prepared all-in-one bundle on GitHub: > > > > > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> > > > > > > > Or you can install it into your own image using
Metacello:
> > > > > > > > Metacellonew > > > > baseline:'TraceDebugger'; > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
> > > > /"repository: > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak > > > > 6.0"/ > > > > get; > > > > load. > > > > > > > > To get started, just open a normal debugger (e.g., by selecting an > > > > expression and pressing Cmd+Shift+D to debug it) and
then
press the > > > > "Trace It" button on the right. There‘s also some
pretty
detailed > > > > documentation in the Help Browser <code://
TraceDebugger
showHelp> > > > > that covers everything you should know. > > > > > > > > My goal is to improve convenience and provide a useful
tool
for the > > > > community, so I‘m very excited to hear your
impressions,
ideas, and > > > > thoughts. Here, on GitHub, or in a private message.
Let‘s
have
a > > > > great discussion! :-) > > > > > > > > Best, > > > > > > > > Christoph (and Marcel) > > > > > > > > PS: Props to Eliot who brought up the original idea of "subclassing > > > > from Context" for other reasons four years ago. [5] > > > > > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
> > > > [2] https://github.com/LinqLover/SimulationStudio > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > > Object-Centric Time-Travel Debugging: Exploring Traces
of
Objects. > > > > https://doi.org/10.1145/3594671.3594678 In
/Companion
Proceedings > > > > of the 7th International Conference on the Art,
Science,
and
> > > > Engineering of Programming/ (/<Programming>'23
Companion/),
March > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
DOI: > > > > 10.1145/3594671.3594678 https://doi.org/10.1145/3594671.3594678. > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
> > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > > Time-Awareness in Object Exploration Tools: Toward In
Situ
> >Omniscient > > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> > > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
on New > > > > Ideas, New Paradigms, and Reflections on Programming
and
Software/ > > > > (/Onward! '23/), October 25–27, 2023, Cascais,
Portugal.
ACM,
New > > > > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > > > https://doi.org/10.1145/3622758.3622892. PDF: > > > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > > > [5] > > > > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > > > > > --- > > > > /Sent from//Squeak Inbox Talk > > > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Hi Jaromir, On 2024-01-04T15:14:55+00:00, mail@jaromir.net wrote:
Hi Christoph, thanks again for your help!
On 04-Jan-24 1:59:00 AM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an
inspector and sometimes I desperately want to compare past and present states. It almost sounds like a preference :)
Hm, I need to think about this ... in the meantime, you can yellow-click an object in a TraceDebugger's/snapshot inspector > basic inspect then yellow-click object > inspect to get its current version. Not very convenient. PRs to TraceDebugger are welcome. :-)
Btw. I usually use the Explorer instead of Inspector but what annoys
me is the Explorer (unlike the Inspector) doesn't update the states automatically: I have to refresh manually (switch to Inspector and back or collapse the hierarchy and open again - is there possibly something to make the Explorer update automatically? Thanks!)
Now you are talking about Squeak in general, right? :) You can yellow-click a node in an explorer and choose "monitor changes". Maybe this should even be the default?
OMG, never noticed... and wasted so much time :D Yes, I'd welcome this behavior as the default (it bit me a few times in the begining before I noticed it wouldn't update in real time) - that would make it consistent with the Inspector's behavior (display 'all inst vars' kind of helps to monitor all fieds but looks a bit untidy).
I think I know why there is no such option: Whenever the monitoring object explorer changes, all child nodes below the root will collapse. If we could handle this, I would be open to making this a preference. :-)
But as you say, snapshots can be helpful in certain situations ... fascinating how a limitation can evolve as a feature ...
(By the way, there is also a preference called "Replace inspectors with explorers in TraceDebugger". But it does not really look nice, that's why its labeled as experimental.)
Cool! For this to look more reasonably you could hide the "value" areas that display the Inspector's values... (screenshot). The debugging is so much more dynamic this way :)
Interesting! But two issues are remaining (at least for me):
(1) You cannot fully read longer values. In inspectors, we have linebreaks and vertical scrolling and the value can use up 60% of the inspector's space. In explorers, we could only allow horizontal scrolling, which is less convenient. (2) Where do you evaluate custom do-its in the debugger? I do this all the time. I guess you could you the main contents pane for this, but then it will always be styled badly, and you cannot see your do-its and longer methods at the same time ...
Understood, thanks. I have noticed, however, that your stepping
methods like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come? Is there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
I would say it's a mixture of both - refactoring and a different model. I put some effort in separating model (TDBCursor) and view (TraceDebugger) from each other better than the Debugger does (so TDBCursor is also usable by itself as a scriptable time-travel debugger). The stepping logic in the normal Debugger is also a bit scattered in Debugger, Process, and Context. Some things are not required in the TraceDebugger, such as the need for the ingenious #runUntilErrorOrReturnFrom: because all contexts during tracing must be simulated.
Ahh, true... is this the source of the performance limitation? I tried to debug some stepOver execution in the Trace debugger and sometimes a step takes a minute or two.
Exactly! If we run certain methods outside of the simulator, we might miss not only parts of the context tree but also changes to variables which I would find even more annoying. Still, I'm aware that this is probably the main limitation of the TraceDebugger, so there is already an open issue for that: https://github.com/hpi-swa-lab/squeak-tracedebugger/issues/106 In a nutshell, users could opt in to skipping certain methods/classes/packages that they consider not relevant for the current debugging session to improve performance (or even make trace debugging of larger programs feasible at all). But this is hard: SimulationStudio must use something like #runUntilErrorOrReturnFrom: to step over irrelevant message sends but reenable the simulation for *every* control flow that exists the context - including exceptions that are eventually resumed and return back into the method stepped over. Same for methods that are activated from within the skipped code that should not be ignored - this would only be possible using breakpoints aka method wrappers. Plus the aforementioned issue of missing state changes. I have been thinking about these issues for a couple of months and my preliminary conclusion is that these requirements go beyond what can be achieved with code simulation. I see greater potential in migrating all the tracing logic away from simulation to method wrappers + bytecode instrumentation, as they have a significantly smaller execution overhead and could be enabled/disabled per method. But that will be a major rewrite ...
Some things also cannot work like in the normal Debugger due to the possibility of stepping backwards, such as the updates to the window label in case of an unhandled error: Normal Debugger uses a Notification for this (see #handleLabelUpdatesIn:whenExecuting:), which only works one-way, whereas the TraceDebugger uses a more functional approach to detect whether the exception handler is still on the stack (see TraceDebugger>>#labelString). Oops, this seems to be broken in the trunk: the "ZeroDivide:" should actually disappear again once after some steps ...
Thanks again!
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-04T00:03:13+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
Thanks a lot for your explanations - very helpful!
a few more notes inlined:
On 02-Jan-24 7:53:31 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev
is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded Collections-ct.1061 to the inbox which would fix it. However, it is
a
bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so
maybe
this needs further discussion.
Or when you open an inspector on a context the context's state
gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows
as
I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might
be
confusing as well because there is no clear visual connection ...
It's
an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an
inspector
and sometimes I desperately want to compare past and present states.
It
almost sounds like a preference :) Btw. I usually use the Explorer instead of Inspector but what annoys me is the Explorer (unlike the Inspector) doesn't update the states automatically: I have to refresh manually (switch to Inspector and back or collapse the hierarchy and open again - is there possibly something to make the Explorer update automatically? Thanks!)
But I'm glad they work well for you. :-) If you have any better
ideas,
let me know!
Regarding your questions about the behavior of code when being run
in
the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am
I
right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
Yes and no. :-) First, the representation of the traced program in
the
TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the
TraceDebugger
with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should)
behave exactly as the same code being run in a normal simulator
(like
when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have been working on fixing these bugs so that all code can behave
exactly
then same when being simulated. Still, there are some open known
(and
likely further unknown) issues (e.g., you cannot simulate a
simulator
which is executing a failed primitive:
ContextrunSimulated:[ContextrunSimulated:[#()tryPrimitive:60withArgs:#(0)]]),
so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not
prepared
to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not
exactly
of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
Understood, thanks. I have noticed, however, that your stepping
methods
like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come? Is there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
However, now you might say: This makes sense when I evaluate Simulatordebug:[thisContextfindNextHandlerContextStarting] because
when
I inspect thisContext in that debugger, it shows a subclass of
Context;
but when I do [thisContextfindNextHandlerContextStarting]debugTrace, thisContext actually is an instance of Context itself, so how can
the
VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but
not
when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContextclass]debugTrace ... The explanation
for
that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short,
we
convert all (non-dead) Context instances from the TraceDebugger's
tree
to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you
with
the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace
debugger
at any point, which will execute the process in the regular VM; and
as
noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulatordebug:[thisContextfindNextHandlerContextStarting]). (Fun
fact:
Not all VMs handle Context subinstances that carefully: SqueakJS
will
seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context,
so
I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Will study, thanks!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context
as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through
again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the
ProcessoractiveProcesssuspend
in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not
yet
honor the suspended/terminated state of the interrupted process.
Maybe
it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am
I
right to expect the Trace debugger behavior would be equivalent
to
the
traditional one?
In other words: if I run the traditional debugger and the Trace
one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I
started
wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context
as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would
be
the
best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would
help
tremendously to be able to ask right away instead of trudging
through
the code/help :) The code usually helps to understand **how**
things
work, the mechanics, but rarely **why**, the intentions.
> you can also turn off the preference "Show call tree in TraceDebugger"
Thanks, that helps to familiarize myself with the new
functionalities
"step-by-step", and not be overwhelmed by all the might of the
call
tree :) Being able to go back is already a hell of an
improvement!
Or
when you open an inspector on a context the context's state gets
frozen
in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the
primitive
call. The Trace debugger, however, starts executing the fallback
code
of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
>Hi Jaromir, > >thanks a lot for trying it out!! Your feedback means a lot to
me.
:-)
> >Indeed you hit a pretty unfortunate example. The TraceDebugger
is
not
>ready for all your clever investigations regarding non-local
returns
>and unwinding. ;-) In fact, your example reveals another
limitation
>that I forgot to mention in the announcement, which regards
programs
>with irregular context switches - e.g., generators/coroutines,
but
>also non-local returns through unwind contexts. This is because
the
>TraceDebugger stores and displays all method invocations in a
tree,
>but in the case of manual context switches, there is no single
global
>tree - its structure would change over the execution time, and
when
>selecting a method invocation, it is not even clear to what
parent
>(sender) it would belong, as there might be multiple. The
current
>solution is to display the tree from the perspective of the
stack
at
>the viewed point in time (see also the '@ <timeIndex>' in the
window
>title), so it looks corrupted while stepping through >Context>>#terminateTo: as the stack is being manipulated. (You
would
>notice the same in a normal debugger if you turned off the
optional
>primitive 196 in this method - for SimulationContexts this
method
>always uses the fallback code.) > >Nevertheless, I have pushed some changes that should allow you
to
step
>out of #terminateTo: again. (You can update the TraceDebugger
from
the
>window menu icon at the right top, like all of my tools.) At
some
>point there will no method be displayed, but you can just step
further
>and eventually return back to the starting point. :-) If you
want
to,
>you can also turn off the preference "Show call tree in
TraceDebugger"
>to make the TraceDebugger look more like a normal debugger,
which
also
>solves the context switches issue. But in general - unless you
are
>debugging unwinding stuff - I would not recommend that as it
removes
>one important strength of the TraceDebugger. :-) > >But again, this is really not a prime example for the
TraceDebugger.
>Better use it to explore how the simulator works. :-) For
example,
you
>could do the following: > >[ContextTest debug: #testBlockCannotReturn] debugTrace. > >And in that trace debugger, you could select the start method,
press
>Cmd + f(ind), and type "return:from:" to investigate the
behavior
of
>your solution there again, etc. > >Thanks for your comments! This was a good chance for me to sort
some
>things out! :-) > >Best, >Christoph > >--- >Sent from Squeak Inbox Talk >https://github.com/hpi-swa-lab/squeak-inbox-talk > >On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote: > > > Hi Christoph, > > > > This indeed sounds like a GREAT idea! I look forward to
seeing
your
>use > > cases to build the right intuition. > > > > In the meantime I've tried to debug/trace this example I've
been
>working > > with lately: > > > > [^2] ensure: [] > > > > If I start the debugger, hit `trace it` and then `step
over`, it
>stops > > at Context>>terminate and the view gets corrupted (the
initial
part
>of > > the trace is hidden and can't be made visible unless
clicking on
>some of > > the pink lines - but not every line does it...) > > > > > > > > > > If I then continue stepping over it ends up with some kind
of
error:
> > > > > > Maybe this is just an unfortunate example... Or maybe I'm
just
doing
> > something wrong... > > > > At any rate - THANKS for your effort!! > > > > > > On 30-Dec-23 4:37:28 PM, >christoph.thiede(a)student.hpi.uni-potsdam.de > > wrote: > > > > >Thanks for the reply, Dave! I will try to post one or two
concrete
>use > > >cases about the TraceDebugger in the next couple of days,
so
stay
> > >tuned. :-) > > > > > >Best, > > >Christoph > > > > > >--- > > >Sent from Squeak Inbox Talk > > >https://github.com/hpi-swa-lab/squeak-inbox-talk > > > > > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote: > > > > > > > This sounds like really interesting work! I love the
idea of
>being > > > > able to interactively go back in "oops, I‘ve stepped too
far,
>let‘s > > > > start all over again" situations. It will probably take
some
>time for > > > > me and others to wrap our heads around the things you
have
done,
>so > > > > don't be surprised if you get a delayed response to this >announcement > > > > :-) > > > > > > > > Congratulations! > > > > Dave > > > > > > > > > > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > > > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > > > > Hi all! > > > > > > > > > > I‘m very excited to announce a project today that we
have
been
> > > > > working on over the past two years: The
*TraceDebugger*
[1] is
>a > > >new > > > > > back-in-time/time-travel/omniscient debugging tool for
Squeak
>that > > > > > allows you to record past method activations and
states
during
> > > > > execution and explore them later. > > > > > > > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > > > > > > > Metacellonew > > > > > baseline:'TraceDebugger'; > > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
> > > > > /"repository: > > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
>Squeak > > > > > 6.0"/ > > > > > get; > > > > > load. > > > > > > > > > > **What can it do? (Features)** > > > > > > > > > > - *Record all method activations and historic states:* >Normally > > >step > > > > > through a program in the debugger while automatically >recording its > > > > > execution. > > > > > - *Replay execution of a traced program:* Navigate
through
all
> > > > > method invocations using the /*context tree*/ or the
/*Step
> > >Back/Step > > > > > Forward*/ buttons (to avoid these "oops, I‘ve stepped
too
far,
> > > > > let‘s start all over again" situations). > > > > > - *Interact with historic states:* Inspect/explore
snapshots
>of > > > > > objects or send them any message. > > > > > - *State-centric debugging using the ***/History >Explorer*/**:* > > > > > Gather, explore, and visualize all changes to an >object/expression > > > > > over the recorded time ("When did this > > >variable/collection/screenshot > > > > > change?"). > > > > > - *Additional navigation tools* for searching and
filtering
>the > > > > > context tree. > > > > > - *Focus on interactivity:* No hours of recording, no
GBs
of
>mem > > > > > consumption - at least for common small to medium
programs.
> > > > > - *UI resembles the classic Smalltalk debugger:*
You'll
find
>your > > > > > familiar stepping buttons, code browsing tools,
inspectors,
>and > > > > > shortcuts - plus more. > > > > > > > > > > The TraceDebugger is a general-purpose tool and not
tied
to
> > > > > particular domains. In the past months, we have
successfully
>used > > >it > > > > > to understand several bugs and interaction patterns in
the
>Trunk > > > > > (Morphic layout/rendering, compiler/decompiler, code >simulation, > > > > > …). The tool is also self-supporting, so you can debug
a
> > > > > TraceDebugger from another TraceDebugger. :-) > > > > > > > > > > **What can‘t it do (yet)? (Limitations and future
work)**
> > > > > > > > > > - *High performance:* While (sufficiently) fast enough
for
>most > > > > > small to medium workloads, tracing very compute- or >mem-intensive > > > > > operations may require more time (ex.:
compiler/decompiler
> > > > > invocation: <1s, HTTPS request: <10s, tool building:
<5m,
>complex > > > > > rendering: minutes up to hours). > > > > > - *Not a dataflow analyzer:* The TraceDebugger does
not
track
> > > > > dataflow events (e.g., argument passing) but only
state
>changes. > > > > > - *No tracing of external states/events* for
FFI/OSProcess
or
> > >custom > > > > > VM modules. > > > > > - *No support for advanced language concepts* such as
identity
> > > > > forwarding/write barriers. > > > > > > > > > > **How does it work? (Implementation)** > > > > > > > > > > In one sentence: To record message sends and side
effects,
we
> > > > > decorate the execution of certain bytecodes with
tracing
>extensions > > > > > by modifying the code simulation using
SimulationStudio
[2].
> > > > > > > > > > In one paragraph: The program is executed in a
specialized
>code > > > > > simulator that overrides instructions for sending
messages
>(e.g., > > > > > send, superSend) and for performing side-effects
(e.g.,
> > >popIntoRcvr, > > > > > primitiveAtPut, push). All message sends are recorded
in a
>tree and > > > > > all changed object slots are stored in a sparse
time-dependent
> > >memory > > > > > structure before they are overwritten. For
time-traveling,
the
>tree > > > > > is traversed using a cursor. For accessing historic
objects, a
> > >proxy > > > > > evaluates all messages sent to an object in another >specialized > > > > > simulator (retracing simulator) that emulates historic
states
>for > > >the > > > > > requested point in time by forwarding read primitives
(e.g.,
> > > > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
>state > > > > > changes in the History Explorer efficiently, the query
is
>evaluated > > > > > in a range retracing simulator with vectorization and
fork
> > >semantics. > > > > > > > > > > In academic terms: We have published two papers about
the
> > > > > TraceDebugger that provide further details about its >implementation > > > > > and its applications for program exploration,
"Object-Centric
> > > > > Time-Travel Debugging: Exploring Traces of Objects"
[3]
and
> > > > > "Time-Awareness in Object Exploration Tools: Toward In
Situ
> > > > > Omniscient Debugging" [4]. > > > > > > > > > > In Smalltalk: Just check out the code base and explore
it
by
> > > > > yourself! The class comments in TraceDebugger > > >code://TraceDebugger > > > > > and TDBCursor code://TDBCursor should provide good
starting
> > >points. > > > > > > > > > > **How can I use it?** > > > > > > > > > > Please try it out and report feedback! The
TraceDebugger
>supports > > > > > the latest Squeak Trunk and Squeak 6.0. You can either >download a > > > > > prepared all-in-one bundle on GitHub: > > > > > > > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> > > > > > > > > > Or you can install it into your own image using
Metacello:
> > > > > > > > > > Metacellonew > > > > > baseline:'TraceDebugger'; > > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
> > > > > /"repository: > > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
>Squeak > > > > > 6.0"/ > > > > > get; > > > > > load. > > > > > > > > > > To get started, just open a normal debugger (e.g., by >selecting an > > > > > expression and pressing Cmd+Shift+D to debug it) and
then
>press the > > > > > "Trace It" button on the right. There‘s also some
pretty
>detailed > > > > > documentation in the Help Browser <code://
TraceDebugger
>showHelp> > > > > > that covers everything you should know. > > > > > > > > > > My goal is to improve convenience and provide a useful
tool
>for the > > > > > community, so I‘m very excited to hear your
impressions,
>ideas, and > > > > > thoughts. Here, on GitHub, or in a private message.
Let‘s
have
>a > > > > > great discussion! :-) > > > > > > > > > > Best, > > > > > > > > > > Christoph (and Marcel) > > > > > > > > > > PS: Props to Eliot who brought up the original idea of >"subclassing > > > > > from Context" for other reasons four years ago. [5] > > > > > > > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
> > > > > [2] https://github.com/LinqLover/SimulationStudio > > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
>2023. > > > > > Object-Centric Time-Travel Debugging: Exploring Traces
of
>Objects. > > > > > https://doi.org/10.1145/3594671.3594678 In
/Companion
>Proceedings > > > > > of the 7th International Conference on the Art,
Science,
and
> > > > > Engineering of Programming/ (/<Programming>'23
Companion/),
>March > > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
>DOI: > > > > > 10.1145/3594671.3594678 >https://doi.org/10.1145/3594671.3594678. > > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
> > > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
>2023. > > > > > Time-Awareness in Object Exploration Tools: Toward In
Situ
> > >Omniscient > > > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> > > > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
>on New > > > > > Ideas, New Paradigms, and Reflections on Programming
and
>Software/ > > > > > (/Onward! '23/), October 25–27, 2023, Cascais,
Portugal.
ACM,
>New > > > > > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > > > > https://doi.org/10.1145/3622758.3622892. PDF: > > > > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > > > > [5] > > > > > > > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > > > > > > > --- > > > > > /Sent from//Squeak Inbox Talk > > > > > https://github.com/hpi-swa-lab/squeak-inbox-talk/
Best, Christoph
--- Sent from Squeak Inbox Talk
Hi Christoph,
On 06-Jan-24 9:51:56 PM, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir, On 2024-01-04T15:14:55+00:00, mail@jaromir.net wrote:
Hi Christoph, thanks again for your help!
On 04-Jan-24 1:59:00 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
[...]
But as you say, snapshots can be helpful in certain situations ... fascinating how a limitation can evolve as a feature ...
(By the way, there is also a preference called "Replace inspectors
with
explorers in TraceDebugger". But it does not really look nice,
that's
why its labeled as experimental.)
Cool! For this to look more reasonably you could hide the "value"
areas
that display the Inspector's values... (screenshot). The debugging is
so
much more dynamic this way :)
Interesting! But two issues are remaining (at least for me):
(1) You cannot fully read longer values. In inspectors, we have linebreaks and vertical scrolling and the value can use up 60% of the inspector's space. In explorers, we could only allow horizontal scrolling, which is less convenient.
True
(2) Where do you evaluate custom do-its in the debugger? I do this all the time. I guess you could you the main contents pane for this, but then it will always be styled badly, and you cannot see your do-its and longer methods at the same time ...
Hmm, main pane usually... but I personally spend more time staring at things than doing them ;)
Thanks again, Jaromir
Understood, thanks. I have noticed, however, that your stepping
methods like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come?
Is
there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
I would say it's a mixture of both - refactoring and a different
model.
I put some effort in separating model (TDBCursor) and view (TraceDebugger) from each other better than the Debugger does (so TDBCursor is also usable by itself as a scriptable time-travel debugger). The stepping logic in the normal Debugger is also a bit scattered in Debugger, Process, and Context. Some things are not required in the TraceDebugger, such as the need for the ingenious #runUntilErrorOrReturnFrom: because all contexts during tracing must
be
simulated.
Ahh, true... is this the source of the performance limitation? I
tried
to debug some stepOver execution in the Trace debugger and sometimes
a
step takes a minute or two.
Exactly! If we run certain methods outside of the simulator, we might miss not only parts of the context tree but also changes to variables which I would find even more annoying. Still, I'm aware that this is probably the main limitation of the TraceDebugger, so there is already an open issue for that: https://github.com/hpi-swa-lab/squeak-tracedebugger/issues/106 In a nutshell, users could opt in to skipping certain methods/classes/packages that they consider not relevant for the current debugging session to improve performance (or even make trace debugging of larger programs feasible at all). But this is hard: SimulationStudio must use something like #runUntilErrorOrReturnFrom: to step over irrelevant message sends but reenable the simulation for *every* control flow that exists the context - including exceptions that are eventually resumed and return back into the method stepped over. Same for methods that are activated from within the skipped code that should not be ignored - this would only be possible using breakpoints aka method wrappers. Plus the aforementioned issue of missing state changes. I have been thinking about these issues for a couple of months and my preliminary conclusion is that these requirements go beyond what can be achieved with code simulation. I see greater potential in migrating all the tracing logic away from simulation to method wrappers + bytecode instrumentation, as they have a significantly smaller execution overhead and could be enabled/disabled per method. But that will be a major rewrite ...
Some things also cannot work like in the normal Debugger due to the possibility of stepping backwards, such as the updates to the window label in case of an unhandled error: Normal Debugger uses a Notification for this (see #handleLabelUpdatesIn:whenExecuting:),
which
only works one-way, whereas the TraceDebugger uses a more functional approach to detect whether the exception handler is still on the
stack
(see TraceDebugger>>#labelString). Oops, this seems to be broken in
the
trunk: the "ZeroDivide:" should actually disappear again once after some steps ...
Thanks again!
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-04T00:03:13+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
Thanks a lot for your explanations - very helpful!
a few more notes inlined:
On 02-Jan-24 7:53:31 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev
is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have
uploaded
Collections-ct.1061 to the inbox which would fix it. However, it
is
a
bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so
maybe
this needs further discussion.
Or when you open an inspector on a context the context's
state
gets
frozen in time and won't change when you proceed debugging -
another
groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers")
have
their pros and cons. As a con, I often found it inconvenient
that I
cannot watch the changing state of certain objects in extra
windows
as
I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger
might
be
confusing as well because there is no clear visual connection
...
It's
an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an
inspector
and sometimes I desperately want to compare past and present
states.
It
almost sounds like a preference :) Btw. I usually use the
Explorer
instead of Inspector but what annoys me is the Explorer (unlike
the
Inspector) doesn't update the states automatically: I have to
refresh
manually (switch to Inspector and back or collapse the hierarchy
and
open again - is there possibly something to make the Explorer
update
automatically? Thanks!)
But I'm glad they work well for you. :-) If you have any better
ideas,
let me know!
Regarding your questions about the behavior of code when being
run
in
the TraceDebugger:
If I turn off the preference "Show call tree in
TraceDebugger" am
I
right to expect the Trace debugger behavior would be equivalent
to
the
traditional one?
Yes and no. :-) First, the representation of the traced program
in
the
TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable
to
locate certain contexts at certain points in time. That's why
these
contexts are skipped as you step through a program in the
TraceDebugger
with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should)
behave exactly as the same code being run in a normal simulator
(like
when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!)
have
been working on fixing these bugs so that all code can behave
exactly
then same when being simulated. Still, there are some open known
(and
likely further unknown) issues (e.g., you cannot simulate a
simulator
which is executing a failed primitive:
ContextrunSimulated:[ContextrunSimulated:[#()tryPrimitive:60withArgs:#(0)]]),
so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when
the
context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio,
which
subclasses from Context (see SimulationContext) to make parts of
the
simulated code execution customizable. The VM, however, is not
prepared
to the existence of such subclass objects of Context and will
always
fail when these primitives are invoked on an object that is not
exactly
of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution
semantics
between normal VM and SimulationStudio/TraceDebugger.
Understood, thanks. I have noticed, however, that your stepping
methods
like stepOver, stepInto and stepThrough are sometimes
substantially
simpler than their traditional counterparts. How come? Is there
some
tradeoff or have you truly refactored and simplified the
traditional
ones? (Which are very opaque and really hard to truly understand)
However, now you might say: This makes sense when I evaluate Simulatordebug:[thisContextfindNextHandlerContextStarting]
because
when
I inspect thisContext in that debugger, it shows a subclass of
Context;
but when I do
[thisContextfindNextHandlerContextStarting]debugTrace,
thisContext actually is an instance of Context itself, so how
can
the
VM detect this? And you would be right, because when you
*inspect* a
context in the TraceDebugger, it is a Context instance indeed,
but
not
when you actually *execute* it in the TraceDebugger, as you can
see
when you evaluate [thisContextclass]debugTrace ... The
explanation
for
that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it
short,
we
convert all (non-dead) Context instances from the
TraceDebugger's
tree
to a subclass of SimulationContext temporarily during each step
to
achieve two things: First, to not confuse observant users like
you
with
the existence of these subclasses (well, maybe that did not work
too
well), and second, to make it possible to resume from a trace
debugger
at any point, which will execute the process in the regular VM;
and
as
noted before, the VM can only handle Context instances, so it
would
fail when scheduling the process otherwise (you can actually
observe
that when trying to proceed from Simulatordebug:[thisContextfindNextHandlerContextStarting]).
(Fun
fact:
Not all VMs handle Context subinstances that carefully: SqueakJS
will
seriously mix up the context/object layout, while TruffleSqueak
will
terminate as soon as you instantiate (!) any subinstance of
Context,
so
I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Will study, thanks!
Example: do step through to the [^2] block and then step
through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context
as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through
again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end
up in
#cannotReturn: when stepping beyond the
ProcessoractiveProcesssuspend
in the bottom context of a process using the TraceDebugger. This
is
because other than the normal debugger, the TraceDebugger does
not
yet
honor the suspended/terminated state of the interrupted process.
Maybe
it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in
TraceDebugger" am
I
right to expect the Trace debugger behavior would be
equivalent
to
the
traditional one?
In other words: if I run the traditional debugger and the
Trace
one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to
simulate
primitive
calls.
However, I've noticed other irregularities so that's why I
started
wondering maybe my assumption was wrong and the Trace
debugger is
designed to present the simulation differently. Please
advise.
Example: do step through to the [^2] block and then step
through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context
as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out
why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
>Hi Christoph, > >Is it ok that I ask questions about the new debugger? What
would
be
the
>best format for such a "Q&A" - here or perhaps within a
topic on
>squeak-smalltalk/squeak-object-memory? I don't expect a
flood of
>questions but to get a bit familiar with your debugger it
would
help
>tremendously to be able to ask right away instead of
trudging
through
>the code/help :) The code usually helps to understand
**how**
things
>work, the mechanics, but rarely **why**, the intentions. > > > you can also turn off the preference "Show call tree in >TraceDebugger" > >Thanks, that helps to familiarize myself with the new
functionalities
>"step-by-step", and not be overwhelmed by all the might of
the
call
>tree :) Being able to go back is already a hell of an
improvement!
Or
>when you open an inspector on a context the context's state
gets
frozen
>in time and won't change when you proceed debugging -
another
>groundbreaking change! > >Question: >In the traditional debugger, when you step into a primitive,
the
>primitive gets executed and the simulation moves over the
primitive
>call. The Trace debugger, however, starts executing the
fallback
code
>of the primitive call - why is that? > >Screenshot after step into #terminateTo: > > > >Thanks again, >Jaromir > >On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
>wrote: > >>Hi Jaromir, >> >>thanks a lot for trying it out!! Your feedback means a lot
to
me.
:-)
>> >>Indeed you hit a pretty unfortunate example. The
TraceDebugger
is
not
>>ready for all your clever investigations regarding
non-local
returns
>>and unwinding. ;-) In fact, your example reveals another
limitation
>>that I forgot to mention in the announcement, which regards
programs
>>with irregular context switches - e.g.,
generators/coroutines,
but
>>also non-local returns through unwind contexts. This is
because
the
>>TraceDebugger stores and displays all method invocations in
a
tree,
>>but in the case of manual context switches, there is no
single
global
>>tree - its structure would change over the execution time,
and
when
>>selecting a method invocation, it is not even clear to what
parent
>>(sender) it would belong, as there might be multiple. The
current
>>solution is to display the tree from the perspective of the
stack
at
>>the viewed point in time (see also the '@ <timeIndex>' in
the
window
>>title), so it looks corrupted while stepping through >>Context>>#terminateTo: as the stack is being manipulated.
(You
would
>>notice the same in a normal debugger if you turned off the
optional
>>primitive 196 in this method - for SimulationContexts this
method
>>always uses the fallback code.) >> >>Nevertheless, I have pushed some changes that should allow
you
to
step
>>out of #terminateTo: again. (You can update the
TraceDebugger
from
the
>>window menu icon at the right top, like all of my tools.)
At
some
>>point there will no method be displayed, but you can just
step
further
>>and eventually return back to the starting point. :-) If
you
want
to,
>>you can also turn off the preference "Show call tree in
TraceDebugger"
>>to make the TraceDebugger look more like a normal debugger,
which
also
>>solves the context switches issue. But in general - unless
you
are
>>debugging unwinding stuff - I would not recommend that as
it
removes
>>one important strength of the TraceDebugger. :-) >> >>But again, this is really not a prime example for the
TraceDebugger.
>>Better use it to explore how the simulator works. :-) For
example,
you
>>could do the following: >> >>[ContextTest debug: #testBlockCannotReturn] debugTrace. >> >>And in that trace debugger, you could select the start
method,
press
>>Cmd + f(ind), and type "return:from:" to investigate the
behavior
of
>>your solution there again, etc. >> >>Thanks for your comments! This was a good chance for me to
sort
some
>>things out! :-) >> >>Best, >>Christoph >> >>--- >>Sent from Squeak Inbox Talk >>https://github.com/hpi-swa-lab/squeak-inbox-talk >> >>On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote: >> >> > Hi Christoph, >> > >> > This indeed sounds like a GREAT idea! I look forward to
seeing
your
>>use >> > cases to build the right intuition. >> > >> > In the meantime I've tried to debug/trace this example
I've
been
>>working >> > with lately: >> > >> > [^2] ensure: [] >> > >> > If I start the debugger, hit `trace it` and then `step
over`, it
>>stops >> > at Context>>terminate and the view gets corrupted (the
initial
part
>>of >> > the trace is hidden and can't be made visible unless
clicking on
>>some of >> > the pink lines - but not every line does it...) >> > >> > >> > >> > >> > If I then continue stepping over it ends up with some
kind
of
error:
>> > >> > >> > Maybe this is just an unfortunate example... Or maybe
I'm
just
doing
>> > something wrong... >> > >> > At any rate - THANKS for your effort!! >> > >> > >> > On 30-Dec-23 4:37:28 PM, >>christoph.thiede(a)student.hpi.uni-potsdam.de >> > wrote: >> > >> > >Thanks for the reply, Dave! I will try to post one or
two
concrete
>>use >> > >cases about the TraceDebugger in the next couple of
days,
so
stay
>> > >tuned. :-) >> > > >> > >Best, >> > >Christoph >> > > >> > >--- >> > >Sent from Squeak Inbox Talk >> > >https://github.com/hpi-swa-lab/squeak-inbox-talk >> > > >> > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com
wrote:
>> > > >> > > > This sounds like really interesting work! I love the
idea of
>>being >> > > > able to interactively go back in "oops, I‘ve stepped
too
far,
>>let‘s >> > > > start all over again" situations. It will probably
take
some
>>time for >> > > > me and others to wrap our heads around the things
you
have
done,
>>so >> > > > don't be surprised if you get a delayed response to
this
>>announcement >> > > > :-) >> > > > >> > > > Congratulations! >> > > > Dave >> > > > >> > > > >> > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, >> > > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: >> > > > > Hi all! >> > > > > >> > > > > I‘m very excited to announce a project today that
we
have
been
>> > > > > working on over the past two years: The
*TraceDebugger*
[1] is
>>a >> > >new >> > > > > back-in-time/time-travel/omniscient debugging tool
for
Squeak
>>that >> > > > > allows you to record past method activations and
states
during
>> > > > > execution and explore them later. >> > > > > >> > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger
>> > > > > >> > > > > Metacellonew >> > > > > baseline:'TraceDebugger'; >> > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
>> > > > > /"repository: >> > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
>>Squeak >> > > > > 6.0"/ >> > > > > get; >> > > > > load. >> > > > > >> > > > > **What can it do? (Features)** >> > > > > >> > > > > - *Record all method activations and historic
states:*
>>Normally >> > >step >> > > > > through a program in the debugger while
automatically
>>recording its >> > > > > execution. >> > > > > - *Replay execution of a traced program:* Navigate
through
all
>> > > > > method invocations using the /*context tree*/ or
the
/*Step
>> > >Back/Step >> > > > > Forward*/ buttons (to avoid these "oops, I‘ve
stepped
too
far,
>> > > > > let‘s start all over again" situations). >> > > > > - *Interact with historic states:* Inspect/explore
snapshots
>>of >> > > > > objects or send them any message. >> > > > > - *State-centric debugging using the ***/History >>Explorer*/**:* >> > > > > Gather, explore, and visualize all changes to an >>object/expression >> > > > > over the recorded time ("When did this >> > >variable/collection/screenshot >> > > > > change?"). >> > > > > - *Additional navigation tools* for searching and
filtering
>>the >> > > > > context tree. >> > > > > - *Focus on interactivity:* No hours of recording,
no
GBs
of
>>mem >> > > > > consumption - at least for common small to medium
programs.
>> > > > > - *UI resembles the classic Smalltalk debugger:*
You'll
find
>>your >> > > > > familiar stepping buttons, code browsing tools,
inspectors,
>>and >> > > > > shortcuts - plus more. >> > > > > >> > > > > The TraceDebugger is a general-purpose tool and
not
tied
to
>> > > > > particular domains. In the past months, we have
successfully
>>used >> > >it >> > > > > to understand several bugs and interaction
patterns in
the
>>Trunk >> > > > > (Morphic layout/rendering, compiler/decompiler,
code
>>simulation, >> > > > > …). The tool is also self-supporting, so you can
debug
a
>> > > > > TraceDebugger from another TraceDebugger. :-) >> > > > > >> > > > > **What can‘t it do (yet)? (Limitations and future
work)**
>> > > > > >> > > > > - *High performance:* While (sufficiently) fast
enough
for
>>most >> > > > > small to medium workloads, tracing very compute-
or
>>mem-intensive >> > > > > operations may require more time (ex.:
compiler/decompiler
>> > > > > invocation: <1s, HTTPS request: <10s, tool
building:
<5m,
>>complex >> > > > > rendering: minutes up to hours). >> > > > > - *Not a dataflow analyzer:* The TraceDebugger
does
not
track
>> > > > > dataflow events (e.g., argument passing) but only
state
>>changes. >> > > > > - *No tracing of external states/events* for
FFI/OSProcess
or
>> > >custom >> > > > > VM modules. >> > > > > - *No support for advanced language concepts* such
as
identity
>> > > > > forwarding/write barriers. >> > > > > >> > > > > **How does it work? (Implementation)** >> > > > > >> > > > > In one sentence: To record message sends and side
effects,
we
>> > > > > decorate the execution of certain bytecodes with
tracing
>>extensions >> > > > > by modifying the code simulation using
SimulationStudio
[2].
>> > > > > >> > > > > In one paragraph: The program is executed in a
specialized
>>code >> > > > > simulator that overrides instructions for sending
messages
>>(e.g., >> > > > > send, superSend) and for performing side-effects
(e.g.,
>> > >popIntoRcvr, >> > > > > primitiveAtPut, push). All message sends are
recorded
in a
>>tree and >> > > > > all changed object slots are stored in a sparse
time-dependent
>> > >memory >> > > > > structure before they are overwritten. For
time-traveling,
the
>>tree >> > > > > is traversed using a cursor. For accessing
historic
objects, a
>> > >proxy >> > > > > evaluates all messages sent to an object in
another
>>specialized >> > > > > simulator (retracing simulator) that emulates
historic
states
>>for >> > >the >> > > > > requested point in time by forwarding read
primitives
(e.g.,
>> > > > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
>>state >> > > > > changes in the History Explorer efficiently, the
query
is
>>evaluated >> > > > > in a range retracing simulator with vectorization
and
fork
>> > >semantics. >> > > > > >> > > > > In academic terms: We have published two papers
about
the
>> > > > > TraceDebugger that provide further details about
its
>>implementation >> > > > > and its applications for program exploration,
"Object-Centric
>> > > > > Time-Travel Debugging: Exploring Traces of
Objects"
[3]
and
>> > > > > "Time-Awareness in Object Exploration Tools:
Toward In
Situ
>> > > > > Omniscient Debugging" [4]. >> > > > > >> > > > > In Smalltalk: Just check out the code base and
explore
it
by
>> > > > > yourself! The class comments in TraceDebugger >> > >code://TraceDebugger >> > > > > and TDBCursor code://TDBCursor should provide
good
starting
>> > >points. >> > > > > >> > > > > **How can I use it?** >> > > > > >> > > > > Please try it out and report feedback! The
TraceDebugger
>>supports >> > > > > the latest Squeak Trunk and Squeak 6.0. You can
either
>>download a >> > > > > prepared all-in-one bundle on GitHub: >> > > > > >> > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
>> > > > > >> > > > > Or you can install it into your own image using
Metacello:
>> > > > > >> > > > > Metacellonew >> > > > > baseline:'TraceDebugger'; >> > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
>> > > > > /"repository: >> > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
>>Squeak >> > > > > 6.0"/ >> > > > > get; >> > > > > load. >> > > > > >> > > > > To get started, just open a normal debugger (e.g.,
by
>>selecting an >> > > > > expression and pressing Cmd+Shift+D to debug it)
and
then
>>press the >> > > > > "Trace It" button on the right. There‘s also some
pretty
>>detailed >> > > > > documentation in the Help Browser <code://
TraceDebugger
>>showHelp> >> > > > > that covers everything you should know. >> > > > > >> > > > > My goal is to improve convenience and provide a
useful
tool
>>for the >> > > > > community, so I‘m very excited to hear your
impressions,
>>ideas, and >> > > > > thoughts. Here, on GitHub, or in a private
message.
Let‘s
have
>>a >> > > > > great discussion! :-) >> > > > > >> > > > > Best, >> > > > > >> > > > > Christoph (and Marcel) >> > > > > >> > > > > PS: Props to Eliot who brought up the original
idea of
>>"subclassing >> > > > > from Context" for other reasons four years ago.
[5]
>> > > > > >> > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
>> > > > > [2]
https://github.com/LinqLover/SimulationStudio
>> > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
>>2023. >> > > > > Object-Centric Time-Travel Debugging: Exploring
Traces
of
>>Objects. >> > > > > https://doi.org/10.1145/3594671.3594678 In
/Companion
>>Proceedings >> > > > > of the 7th International Conference on the Art,
Science,
and
>> > > > > Engineering of Programming/ (/<Programming>'23
Companion/),
>>March >> > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA,
7
pages.
>>DOI: >> > > > > 10.1145/3594671.3594678 >>https://doi.org/10.1145/3594671.3594678. >> > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
>> > > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
>>2023. >> > > > > Time-Awareness in Object Exploration Tools: Toward
In
Situ
>> > >Omniscient >> > > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
>> > > > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
>>on New >> > > > > Ideas, New Paradigms, and Reflections on
Programming
and
>>Software/ >> > > > > (/Onward! '23/), October 25–27, 2023, Cascais,
Portugal.
ACM,
>>New >> > > > > York, NY, USA, 14 pages. DOI:
10.1145/3622758.3622892
>> > > > > https://doi.org/10.1145/3622758.3622892. PDF: >> > > > >
https://dl.acm.org/doi/pdf/10.1145/3622758.3622892
>> > > > > [5] >> > > > > >> > >>
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
>> > > > > >> > > > > --- >> > > > > /Sent from//Squeak Inbox Talk >> > > > >
https://github.com/hpi-swa-lab/squeak-inbox-talk/
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
Hi Jaromir,
On 2024-01-06T23:23:36+00:00, mail@jaromir.net wrote:
Hi Christoph,
On 06-Jan-24 9:51:56 PM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir, On 2024-01-04T15:14:55+00:00, mail(a)jaromir.net wrote:
Hi Christoph, thanks again for your help!
On 04-Jan-24 1:59:00 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
[...]
But as you say, snapshots can be helpful in certain situations ... fascinating how a limitation can evolve as a feature ...
(By the way, there is also a preference called "Replace inspectors
with
explorers in TraceDebugger". But it does not really look nice,
that's
why its labeled as experimental.)
Cool! For this to look more reasonably you could hide the "value"
areas
that display the Inspector's values... (screenshot). The debugging is
so
much more dynamic this way :)
Interesting! But two issues are remaining (at least for me):
(1) You cannot fully read longer values. In inspectors, we have linebreaks and vertical scrolling and the value can use up 60% of the inspector's space. In explorers, we could only allow horizontal scrolling, which is less convenient.
True
(2) Where do you evaluate custom do-its in the debugger? I do this all the time. I guess you could you the main contents pane for this, but then it will always be styled badly, and you cannot see your do-its and longer methods at the same time ...
Hmm, main pane usually... but I personally spend more time staring at things than doing them ;)
I also do this sometimes. But mainly my philosophy of Squeak is that you are having vivid conversations with the system and many of its objects all the time. Don't assume things, but ask the system straight up about it! (What is your home method? What instruction will you execute next? Have you already been terminated? How do you simulate that primitive? And with the TraceDebugger also: What have you done before? When/why did your program counter change?) Having that direct access to objects through print-its/debug-its is a luxury that's still not available in many mainstream languages. But sometimes, our tools are not yet capable or intuitive enough to help with this - e.g., we cannot see what happens inside the VM, we don't have good tools for debugging (or tracing) multiple processes, and the interfaces that we've ve got for expressing and answering our questions do not always align directly to our mental model, thus making question answering harder than "staring" and hypothetizing by yourself. That's just my "OO-extremist" perspective, of course. :-)
Best, Christoph
Thanks again, Jaromir
Understood, thanks. I have noticed, however, that your stepping
methods like stepOver, stepInto and stepThrough are sometimes substantially simpler than their traditional counterparts. How come?
Is
there some tradeoff or have you truly refactored and simplified the traditional ones? (Which are very opaque and really hard to truly understand)
I would say it's a mixture of both - refactoring and a different
model.
I put some effort in separating model (TDBCursor) and view (TraceDebugger) from each other better than the Debugger does (so TDBCursor is also usable by itself as a scriptable time-travel debugger). The stepping logic in the normal Debugger is also a bit scattered in Debugger, Process, and Context. Some things are not required in the TraceDebugger, such as the need for the ingenious #runUntilErrorOrReturnFrom: because all contexts during tracing must
be
simulated.
Ahh, true... is this the source of the performance limitation? I
tried
to debug some stepOver execution in the Trace debugger and sometimes
a
step takes a minute or two.
Exactly! If we run certain methods outside of the simulator, we might miss not only parts of the context tree but also changes to variables which I would find even more annoying. Still, I'm aware that this is probably the main limitation of the TraceDebugger, so there is already an open issue for that: https://github.com/hpi-swa-lab/squeak-tracedebugger/issues/106 In a nutshell, users could opt in to skipping certain methods/classes/packages that they consider not relevant for the current debugging session to improve performance (or even make trace debugging of larger programs feasible at all). But this is hard: SimulationStudio must use something like #runUntilErrorOrReturnFrom: to step over irrelevant message sends but reenable the simulation for *every* control flow that exists the context - including exceptions that are eventually resumed and return back into the method stepped over. Same for methods that are activated from within the skipped code that should not be ignored - this would only be possible using breakpoints aka method wrappers. Plus the aforementioned issue of missing state changes. I have been thinking about these issues for a couple of months and my preliminary conclusion is that these requirements go beyond what can be achieved with code simulation. I see greater potential in migrating all the tracing logic away from simulation to method wrappers + bytecode instrumentation, as they have a significantly smaller execution overhead and could be enabled/disabled per method. But that will be a major rewrite ...
Some things also cannot work like in the normal Debugger due to the possibility of stepping backwards, such as the updates to the window label in case of an unhandled error: Normal Debugger uses a Notification for this (see #handleLabelUpdatesIn:whenExecuting:),
which
only works one-way, whereas the TraceDebugger uses a more functional approach to detect whether the exception handler is still on the
stack
(see TraceDebugger>>#labelString). Oops, this seems to be broken in
the
trunk: the "ZeroDivide:" should actually disappear again once after some steps ...
Thanks again!
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-04T00:03:13+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
Thanks a lot for your explanations - very helpful!
a few more notes inlined:
On 02-Jan-24 7:53:31 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev
is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have
uploaded
Collections-ct.1061 to the inbox which would fix it. However, it
is
a
bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so
maybe
this needs further discussion.
> Or when you open an inspector on a context the context's
state
gets
frozen in time and won't change when you proceed debugging -
another
groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers")
have
their pros and cons. As a con, I often found it inconvenient
that I
cannot watch the changing state of certain objects in extra
windows
as
I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger
might
be
confusing as well because there is no clear visual connection
...
It's
an unsolved UX problem for me.
It depends: sometimes I like to see the states changing in an
inspector
and sometimes I desperately want to compare past and present
states.
It
almost sounds like a preference :) Btw. I usually use the
Explorer
instead of Inspector but what annoys me is the Explorer (unlike
the
Inspector) doesn't update the states automatically: I have to
refresh
manually (switch to Inspector and back or collapse the hierarchy
and
open again - is there possibly something to make the Explorer
update
automatically? Thanks!)
But I'm glad they work well for you. :-) If you have any better
ideas,
let me know!
Regarding your questions about the behavior of code when being
run
in
the TraceDebugger:
> If I turn off the preference "Show call tree in
TraceDebugger" am
I
right to expect the Trace debugger behavior would be equivalent
to
the
traditional one?
Yes and no. :-) First, the representation of the traced program
in
the
TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable
to
locate certain contexts at certain points in time. That's why
these
contexts are skipped as you step through a program in the
TraceDebugger
with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should)
behave exactly as the same code being run in a normal simulator
(like
when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!)
have
been working on fixing these bugs so that all code can behave
exactly
then same when being simulated. Still, there are some open known
(and
likely further unknown) issues (e.g., you cannot simulate a
simulator
which is executing a failed primitive:
ContextrunSimulated:[ContextrunSimulated:[#()tryPrimitive:60withArgs:#(0)]]),
so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when
the
context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio,
which
subclasses from Context (see SimulationContext) to make parts of
the
simulated code execution customizable. The VM, however, is not
prepared
to the existence of such subclass objects of Context and will
always
fail when these primitives are invoked on an object that is not
exactly
of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution
semantics
between normal VM and SimulationStudio/TraceDebugger.
Understood, thanks. I have noticed, however, that your stepping
methods
like stepOver, stepInto and stepThrough are sometimes
substantially
simpler than their traditional counterparts. How come? Is there
some
tradeoff or have you truly refactored and simplified the
traditional
ones? (Which are very opaque and really hard to truly understand)
However, now you might say: This makes sense when I evaluate Simulatordebug:[thisContextfindNextHandlerContextStarting]
because
when
I inspect thisContext in that debugger, it shows a subclass of
Context;
but when I do
[thisContextfindNextHandlerContextStarting]debugTrace,
thisContext actually is an instance of Context itself, so how
can
the
VM detect this? And you would be right, because when you
*inspect* a
context in the TraceDebugger, it is a Context instance indeed,
but
not
when you actually *execute* it in the TraceDebugger, as you can
see
when you evaluate [thisContextclass]debugTrace ... The
explanation
for
that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it
short,
we
convert all (non-dead) Context instances from the
TraceDebugger's
tree
to a subclass of SimulationContext temporarily during each step
to
achieve two things: First, to not confuse observant users like
you
with
the existence of these subclasses (well, maybe that did not work
too
well), and second, to make it possible to resume from a trace
debugger
at any point, which will execute the process in the regular VM;
and
as
noted before, the VM can only handle Context instances, so it
would
fail when scheduling the process otherwise (you can actually
observe
that when trying to proceed from Simulatordebug:[thisContextfindNextHandlerContextStarting]).
(Fun
fact:
Not all VMs handle Context subinstances that carefully: SqueakJS
will
seriously mix up the context/object layout, while TruffleSqueak
will
terminate as soon as you instantiate (!) any subinstance of
Context,
so
I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Will study, thanks!
> Example: do step through to the [^2] block and then step
through
again > > [^2] ensure: [] > > Traditionally, you end up in the unwind block. > In the new Trace debugger you end up with the #cannotReturn:
context
as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through
again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end
up in
#cannotReturn: when stepping beyond the
ProcessoractiveProcesssuspend
in the bottom context of a process using the TraceDebugger. This
is
because other than the normal debugger, the TraceDebugger does
not
yet
honor the suspended/terminated state of the interrupted process.
Maybe
it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
> Hi Christoph, > > sorry, a follow-up question :) > > If I turn off the preference "Show call tree in
TraceDebugger" am
I
> right to expect the Trace debugger behavior would be
equivalent
to
the > traditional one? > > In other words: if I run the traditional debugger and the
Trace
one
side > by side, should they display analogous steps? > > In the other message I wrote about a different way to
simulate
primitive > calls. > > However, I've noticed other irregularities so that's why I
started
> wondering maybe my assumption was wrong and the Trace
debugger is
> designed to present the simulation differently. Please
advise.
> > Example: do step through to the [^2] block and then step
through
again > > [^2] ensure: [] > > Traditionally, you end up in the unwind block. > In the new Trace debugger you end up with the #cannotReturn:
context
as > if the computation just ran until the end. > > Is this expected? (My guess is it isn't but can't figure out
why)
> > Thanks again, > Jaromir > > > > > > On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
> > >Hi Christoph, > > > >Is it ok that I ask questions about the new debugger? What
would
be
the > >best format for such a "Q&A" - here or perhaps within a
topic on
> >squeak-smalltalk/squeak-object-memory? I don't expect a
flood of
> >questions but to get a bit familiar with your debugger it
would
help
> >tremendously to be able to ask right away instead of
trudging
through > >the code/help :) The code usually helps to understand
**how**
things
> >work, the mechanics, but rarely **why**, the intentions. > > > > > you can also turn off the preference "Show call tree in > >TraceDebugger" > > > >Thanks, that helps to familiarize myself with the new functionalities > >"step-by-step", and not be overwhelmed by all the might of
the
call
> >tree :) Being able to go back is already a hell of an
improvement!
Or > >when you open an inspector on a context the context's state
gets
frozen > >in time and won't change when you proceed debugging -
another
> >groundbreaking change! > > > >Question: > >In the traditional debugger, when you step into a primitive,
the
> >primitive gets executed and the simulation moves over the
primitive
> >call. The Trace debugger, however, starts executing the
fallback
code > >of the primitive call - why is that? > > > >Screenshot after step into #terminateTo: > > > > > > > >Thanks again, > >Jaromir > > > >On 31-Dec-23 2:16:32 AM, christoph.thiede(a)student.hpi.uni-potsdam.de > >wrote: > > > >>Hi Jaromir, > >> > >>thanks a lot for trying it out!! Your feedback means a lot
to
me.
:-) > >> > >>Indeed you hit a pretty unfortunate example. The
TraceDebugger
is
not > >>ready for all your clever investigations regarding
non-local
returns > >>and unwinding. ;-) In fact, your example reveals another
limitation
> >>that I forgot to mention in the announcement, which regards programs > >>with irregular context switches - e.g.,
generators/coroutines,
but
> >>also non-local returns through unwind contexts. This is
because
the
> >>TraceDebugger stores and displays all method invocations in
a
tree,
> >>but in the case of manual context switches, there is no
single
global > >>tree - its structure would change over the execution time,
and
when
> >>selecting a method invocation, it is not even clear to what
parent
> >>(sender) it would belong, as there might be multiple. The
current
> >>solution is to display the tree from the perspective of the
stack
at > >>the viewed point in time (see also the '@ <timeIndex>' in
the
window > >>title), so it looks corrupted while stepping through > >>Context>>#terminateTo: as the stack is being manipulated.
(You
would > >>notice the same in a normal debugger if you turned off the
optional
> >>primitive 196 in this method - for SimulationContexts this
method
> >>always uses the fallback code.) > >> > >>Nevertheless, I have pushed some changes that should allow
you
to
step > >>out of #terminateTo: again. (You can update the
TraceDebugger
from
the > >>window menu icon at the right top, like all of my tools.)
At
some
> >>point there will no method be displayed, but you can just
step
further > >>and eventually return back to the starting point. :-) If
you
want
to, > >>you can also turn off the preference "Show call tree in TraceDebugger" > >>to make the TraceDebugger look more like a normal debugger,
which
also > >>solves the context switches issue. But in general - unless
you
are
> >>debugging unwinding stuff - I would not recommend that as
it
removes > >>one important strength of the TraceDebugger. :-) > >> > >>But again, this is really not a prime example for the TraceDebugger. > >>Better use it to explore how the simulator works. :-) For
example,
you > >>could do the following: > >> > >>[ContextTest debug: #testBlockCannotReturn] debugTrace. > >> > >>And in that trace debugger, you could select the start
method,
press > >>Cmd + f(ind), and type "return:from:" to investigate the
behavior
of > >>your solution there again, etc. > >> > >>Thanks for your comments! This was a good chance for me to
sort
some > >>things out! :-) > >> > >>Best, > >>Christoph > >> > >>--- > >>Sent from Squeak Inbox Talk > >>https://github.com/hpi-swa-lab/squeak-inbox-talk > >> > >>On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote: > >> > >> > Hi Christoph, > >> > > >> > This indeed sounds like a GREAT idea! I look forward to
seeing
your > >>use > >> > cases to build the right intuition. > >> > > >> > In the meantime I've tried to debug/trace this example
I've
been
> >>working > >> > with lately: > >> > > >> > [^2] ensure: [] > >> > > >> > If I start the debugger, hit `trace it` and then `step
over`, it
> >>stops > >> > at Context>>terminate and the view gets corrupted (the
initial
part > >>of > >> > the trace is hidden and can't be made visible unless
clicking on
> >>some of > >> > the pink lines - but not every line does it...) > >> > > >> > > >> > > >> > > >> > If I then continue stepping over it ends up with some
kind
of
error: > >> > > >> > > >> > Maybe this is just an unfortunate example... Or maybe
I'm
just
doing > >> > something wrong... > >> > > >> > At any rate - THANKS for your effort!! > >> > > >> > > >> > On 30-Dec-23 4:37:28 PM, > >>christoph.thiede(a)student.hpi.uni-potsdam.de > >> > wrote: > >> > > >> > >Thanks for the reply, Dave! I will try to post one or
two
concrete > >>use > >> > >cases about the TraceDebugger in the next couple of
days,
so
stay > >> > >tuned. :-) > >> > > > >> > >Best, > >> > >Christoph > >> > > > >> > >--- > >> > >Sent from Squeak Inbox Talk > >> > >https://github.com/hpi-swa-lab/squeak-inbox-talk > >> > > > >> > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com
wrote:
> >> > > > >> > > > This sounds like really interesting work! I love the
idea of
> >>being > >> > > > able to interactively go back in "oops, I‘ve stepped
too
far, > >>let‘s > >> > > > start all over again" situations. It will probably
take
some
> >>time for > >> > > > me and others to wrap our heads around the things
you
have
done, > >>so > >> > > > don't be surprised if you get a delayed response to
this
> >>announcement > >> > > > :-) > >> > > > > >> > > > Congratulations! > >> > > > Dave > >> > > > > >> > > > > >> > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > >> > > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > >> > > > > Hi all! > >> > > > > > >> > > > > I‘m very excited to announce a project today that
we
have
been > >> > > > > working on over the past two years: The
*TraceDebugger*
[1] is > >>a > >> > >new > >> > > > > back-in-time/time-travel/omniscient debugging tool
for
Squeak > >>that > >> > > > > allows you to record past method activations and
states
during > >> > > > > execution and explore them later. > >> > > > > > >> > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger
> >> > > > > > >> > > > > Metacellonew > >> > > > > baseline:'TraceDebugger'; > >> > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
> >> > > > > /"repository: > >> > > > > 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for > >>Squeak > >> > > > > 6.0"/ > >> > > > > get; > >> > > > > load. > >> > > > > > >> > > > > **What can it do? (Features)** > >> > > > > > >> > > > > - *Record all method activations and historic
states:*
> >>Normally > >> > >step > >> > > > > through a program in the debugger while
automatically
> >>recording its > >> > > > > execution. > >> > > > > - *Replay execution of a traced program:* Navigate
through
all > >> > > > > method invocations using the /*context tree*/ or
the
/*Step > >> > >Back/Step > >> > > > > Forward*/ buttons (to avoid these "oops, I‘ve
stepped
too
far, > >> > > > > let‘s start all over again" situations). > >> > > > > - *Interact with historic states:* Inspect/explore snapshots > >>of > >> > > > > objects or send them any message. > >> > > > > - *State-centric debugging using the ***/History > >>Explorer*/**:* > >> > > > > Gather, explore, and visualize all changes to an > >>object/expression > >> > > > > over the recorded time ("When did this > >> > >variable/collection/screenshot > >> > > > > change?"). > >> > > > > - *Additional navigation tools* for searching and filtering > >>the > >> > > > > context tree. > >> > > > > - *Focus on interactivity:* No hours of recording,
no
GBs
of > >>mem > >> > > > > consumption - at least for common small to medium programs. > >> > > > > - *UI resembles the classic Smalltalk debugger:*
You'll
find > >>your > >> > > > > familiar stepping buttons, code browsing tools, inspectors, > >>and > >> > > > > shortcuts - plus more. > >> > > > > > >> > > > > The TraceDebugger is a general-purpose tool and
not
tied
to > >> > > > > particular domains. In the past months, we have successfully > >>used > >> > >it > >> > > > > to understand several bugs and interaction
patterns in
the
> >>Trunk > >> > > > > (Morphic layout/rendering, compiler/decompiler,
code
> >>simulation, > >> > > > > …). The tool is also self-supporting, so you can
debug
a
> >> > > > > TraceDebugger from another TraceDebugger. :-) > >> > > > > > >> > > > > **What can‘t it do (yet)? (Limitations and future
work)**
> >> > > > > > >> > > > > - *High performance:* While (sufficiently) fast
enough
for
> >>most > >> > > > > small to medium workloads, tracing very compute-
or
> >>mem-intensive > >> > > > > operations may require more time (ex.:
compiler/decompiler
> >> > > > > invocation: <1s, HTTPS request: <10s, tool
building:
<5m,
> >>complex > >> > > > > rendering: minutes up to hours). > >> > > > > - *Not a dataflow analyzer:* The TraceDebugger
does
not
track > >> > > > > dataflow events (e.g., argument passing) but only
state
> >>changes. > >> > > > > - *No tracing of external states/events* for
FFI/OSProcess
or > >> > >custom > >> > > > > VM modules. > >> > > > > - *No support for advanced language concepts* such
as
identity > >> > > > > forwarding/write barriers. > >> > > > > > >> > > > > **How does it work? (Implementation)** > >> > > > > > >> > > > > In one sentence: To record message sends and side
effects,
we > >> > > > > decorate the execution of certain bytecodes with
tracing
> >>extensions > >> > > > > by modifying the code simulation using
SimulationStudio
[2]. > >> > > > > > >> > > > > In one paragraph: The program is executed in a
specialized
> >>code > >> > > > > simulator that overrides instructions for sending
messages
> >>(e.g., > >> > > > > send, superSend) and for performing side-effects
(e.g.,
> >> > >popIntoRcvr, > >> > > > > primitiveAtPut, push). All message sends are
recorded
in a
> >>tree and > >> > > > > all changed object slots are stored in a sparse time-dependent > >> > >memory > >> > > > > structure before they are overwritten. For
time-traveling,
the > >>tree > >> > > > > is traversed using a cursor. For accessing
historic
objects, a > >> > >proxy > >> > > > > evaluates all messages sent to an object in
another
> >>specialized > >> > > > > simulator (retracing simulator) that emulates
historic
states > >>for > >> > >the > >> > > > > requested point in time by forwarding read
primitives
(e.g., > >> > > > > pushRcvr, primitiveAt) to the recorded memory. For gathering > >>state > >> > > > > changes in the History Explorer efficiently, the
query
is
> >>evaluated > >> > > > > in a range retracing simulator with vectorization
and
fork
> >> > >semantics. > >> > > > > > >> > > > > In academic terms: We have published two papers
about
the
> >> > > > > TraceDebugger that provide further details about
its
> >>implementation > >> > > > > and its applications for program exploration, "Object-Centric > >> > > > > Time-Travel Debugging: Exploring Traces of
Objects"
[3]
and > >> > > > > "Time-Awareness in Object Exploration Tools:
Toward In
Situ > >> > > > > Omniscient Debugging" [4]. > >> > > > > > >> > > > > In Smalltalk: Just check out the code base and
explore
it
by > >> > > > > yourself! The class comments in TraceDebugger > >> > >code://TraceDebugger > >> > > > > and TDBCursor code://TDBCursor should provide
good
starting > >> > >points. > >> > > > > > >> > > > > **How can I use it?** > >> > > > > > >> > > > > Please try it out and report feedback! The
TraceDebugger
> >>supports > >> > > > > the latest Squeak Trunk and Squeak 6.0. You can
either
> >>download a > >> > > > > prepared all-in-one bundle on GitHub: > >> > > > > > >> > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger/releases > >> > > > > > >> > > > > Or you can install it into your own image using
Metacello:
> >> > > > > > >> > > > > Metacellonew > >> > > > > baseline:'TraceDebugger'; > >> > > > >
repository:'github://hpi-swa-lab/squeak-tracedebugger';
> >> > > > > /"repository: > >> > > > > 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for > >>Squeak > >> > > > > 6.0"/ > >> > > > > get; > >> > > > > load. > >> > > > > > >> > > > > To get started, just open a normal debugger (e.g.,
by
> >>selecting an > >> > > > > expression and pressing Cmd+Shift+D to debug it)
and
then
> >>press the > >> > > > > "Trace It" button on the right. There‘s also some
pretty
> >>detailed > >> > > > > documentation in the Help Browser <code://
TraceDebugger
> >>showHelp> > >> > > > > that covers everything you should know. > >> > > > > > >> > > > > My goal is to improve convenience and provide a
useful
tool > >>for the > >> > > > > community, so I‘m very excited to hear your
impressions,
> >>ideas, and > >> > > > > thoughts. Here, on GitHub, or in a private
message.
Let‘s
have > >>a > >> > > > > great discussion! :-) > >> > > > > > >> > > > > Best, > >> > > > > > >> > > > > Christoph (and Marcel) > >> > > > > > >> > > > > PS: Props to Eliot who brought up the original
idea of
> >>"subclassing > >> > > > > from Context" for other reasons four years ago.
[5]
> >> > > > > > >> > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
> >> > > > > [2]
https://github.com/LinqLover/SimulationStudio
> >> > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. > >>2023. > >> > > > > Object-Centric Time-Travel Debugging: Exploring
Traces
of
> >>Objects. > >> > > > > https://doi.org/10.1145/3594671.3594678 In
/Companion
> >>Proceedings > >> > > > > of the 7th International Conference on the Art,
Science,
and > >> > > > > Engineering of Programming/ (/<Programming>'23 Companion/), > >>March > >> > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA,
7
pages. > >>DOI: > >> > > > > 10.1145/3594671.3594678 > >>https://doi.org/10.1145/3594671.3594678. > >> > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
> >> > > > > [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. > >>2023. > >> > > > > Time-Awareness in Object Exploration Tools: Toward
In
Situ
> >> > >Omniscient > >> > > > > Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In > >> > > > > /Proceedings of the 2023 ACM SIGPLAN International Symposium > >>on New > >> > > > > Ideas, New Paradigms, and Reflections on
Programming
and
> >>Software/ > >> > > > > (/Onward! '23/), October 25–27, 2023, Cascais,
Portugal.
ACM, > >>New > >> > > > > York, NY, USA, 14 pages. DOI:
10.1145/3622758.3622892
> >> > > > > https://doi.org/10.1145/3622758.3622892. PDF: > >> > > > >
https://dl.acm.org/doi/pdf/10.1145/3622758.3622892
> >> > > > > [5] > >> > > > > > >> > > >>
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> >> > > > > > >> > > > > --- > >> > > > > /Sent from//Squeak Inbox Talk > >> > > > >
https://github.com/hpi-swa-lab/squeak-inbox-talk/
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
--- Sent from Squeak Inbox Talk
Congratulations on crossing the finish line of this project / milestone! This looks really interesting. I gleaned from merely the abstract (Object-Centric Time-Travel paper) that the goal is not merely the novel convenience of backward stepping, but to facilitate a deep understanding of how a large and/or complex application works, or why a particular bug exists that otherwise isn't easy to understand (i.e., UI debugging).
One question: Is the changed-detection / response mechanism usable with domains outside of trace debugging? Or is it (understandably) hard-wired to work only within the context of trace debugging?
What a great round out for 2023! The quality and quantity of mind-blowing projects from this community is mind-blowing. I'm holding up my glass in (virtual) toast to Squeak for this past and coming year.
- Chris
On Thu, Dec 28, 2023 at 6:42 PM christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi all!
I‘m very excited to announce a project today that we have been working on over the past two years: The *TraceDebugger* [1] is a new back-in-time/time-travel/omniscient debugging tool for Squeak that allows you to record past method activations and states during execution and explore them later.
https://github.com/hpi-swa-lab/squeak-tracedebugger
Metacello new baseline: 'TraceDebugger'; repository: 'github://hpi-swa-lab/squeak-tracedebugger'; *"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"* *"for Squeak 6.0"* get; load.
**What can it do? (Features)**
- *Record all method activations and historic states:* Normally step
through a program in the debugger while automatically recording its execution.
- *Replay execution of a traced program:* Navigate through all method
invocations using the **context tree** or the **Step Back/Step Forward** buttons (to avoid these "oops, I‘ve stepped too far, let‘s start all over again" situations).
- *Interact with historic states:* Inspect/explore snapshots of objects
or send them any message.
- *State-centric debugging using the ***History Explorer***:* Gather,
explore, and visualize all changes to an object/expression over the recorded time ("When did this variable/collection/screenshot change?").
- *Additional navigation tools* for searching and filtering the context
tree.
- *Focus on interactivity:* No hours of recording, no GBs of mem
consumption - at least for common small to medium programs.
- *UI resembles the classic Smalltalk debugger:* You'll find your
familiar stepping buttons, code browsing tools, inspectors, and shortcuts - plus more.
The TraceDebugger is a general-purpose tool and not tied to particular domains. In the past months, we have successfully used it to understand several bugs and interaction patterns in the Trunk (Morphic layout/rendering, compiler/decompiler, code simulation, …). The tool is also self-supporting, so you can debug a TraceDebugger from another TraceDebugger. :-)
**What can‘t it do (yet)? (Limitations and future work)**
- *High performance:* While (sufficiently) fast enough for most small to
medium workloads, tracing very compute- or mem-intensive operations may require more time (ex.: compiler/decompiler invocation: <1s, HTTPS request: <10s, tool building: <5m, complex rendering: minutes up to hours).
- *Not a dataflow analyzer:* The TraceDebugger does not track dataflow
events (e.g., argument passing) but only state changes.
- *No tracing of external states/events* for FFI/OSProcess or custom VM
modules.
- *No support for advanced language concepts* such as identity
forwarding/write barriers.
**How does it work? (Implementation)**
In one sentence: To record message sends and side effects, we decorate the execution of certain bytecodes with tracing extensions by modifying the code simulation using SimulationStudio [2].
In one paragraph: The program is executed in a specialized code simulator that overrides instructions for sending messages (e.g., send, superSend) and for performing side-effects (e.g., popIntoRcvr, primitiveAtPut, push). All message sends are recorded in a tree and all changed object slots are stored in a sparse time-dependent memory structure before they are overwritten. For time-traveling, the tree is traversed using a cursor. For accessing historic objects, a proxy evaluates all messages sent to an object in another specialized simulator (retracing simulator) that emulates historic states for the requested point in time by forwarding read primitives (e.g., pushRcvr, primitiveAt) to the recorded memory. For gathering state changes in the History Explorer efficiently, the query is evaluated in a range retracing simulator with vectorization and fork semantics.
In academic terms: We have published two papers about the TraceDebugger that provide further details about its implementation and its applications for program exploration, "Object-Centric Time-Travel Debugging: Exploring Traces of Objects" [3] and "Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging" [4].
In Smalltalk: Just check out the code base and explore it by yourself! The class comments in TraceDebugger and TDBCursor should provide good starting points.
**How can I use it?**
Please try it out and report feedback! The TraceDebugger supports the latest Squeak Trunk and Squeak 6.0. You can either download a prepared all-in-one bundle on GitHub:
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
Or you can install it into your own image using Metacello:
Metacello new baseline: 'TraceDebugger'; repository: 'github://hpi-swa-lab/squeak-tracedebugger'; *"repository: 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"* *"for Squeak 6.0"* get; load.
To get started, just open a normal debugger (e.g., by selecting an expression and pressing Cmd+Shift+D to debug it) and then press the "Trace It" button on the right. There‘s also some pretty detailed documentation in the Help Browser that covers everything you should know.
My goal is to improve convenience and provide a useful tool for the community, so I‘m very excited to hear your impressions, ideas, and thoughts. Here, on GitHub, or in a private message. Let‘s have a great discussion! :-)
Best,
Christoph (and Marcel)
PS: Props to Eliot who brought up the original idea of "subclassing from Context" for other reasons four years ago. [5]
[1] https://github.com/hpi-swa-lab/squeak-tracedebugger [2] https://github.com/LinqLover/SimulationStudio [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Object-Centric Time-Travel Debugging: Exploring Traces of Objects. https://doi.org/10.1145/3594671.3594678 In *Companion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming* (*<Programming>'23 Companion*), March 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages. DOI: 10.1145/3594671.3594678 https://doi.org/10.1145/3594671.3594678. PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld. 2023. Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In *Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software* (*Onward! '23*), October 25–27, 2023, Cascais, Portugal. ACM, New York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 https://doi.org/10.1145/3622758.3622892. PDF: https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 [5] http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.h...
*Sent from **Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk*
Hi Chris,
On 2023-12-31T16:42:58-06:00, asqueaker@gmail.com wrote:
Congratulations on crossing the finish line of this project / milestone! This looks really interesting. I gleaned from merely the abstract (Object-Centric Time-Travel paper) that the goal is not merely the novel convenience of backward stepping, but to facilitate a deep understanding of how a large and/or complex application works, or why a particular bug exists that otherwise isn't easy to understand (i.e., UI debugging).
Very nicely put!
One question: Is the changed-detection / response mechanism usable with domains outside of trace debugging? Or is it (understandably) hard-wired to work only within the context of trace debugging?
In a nutshell, the change detection in the TraceDebugger works by simulating all the code using the simulation protocol in the Context class and overriding single primitives/instructions there that perform side effects to log their effect. So while in theory you could also use that to trigger events immediately when a change is detected, this would in practice slow down the execution by >100,000% (sic!). So, while it is fine to run small to medium expressions in such a simulator, simulating all the image the entire time is likely not fun*.
However, if you are interested in such a mechanism, here are two related approaches:
* Object>>#evaluate:wheneverChangeIn: is ingenious and works surprising well in the trunk. I even use it somewhere in "production" code for one of my side projects. * Active expressions are a similar mechanism to integrate arbitrary side effect watchers into the language. Here is an older implementation for Squeak (I have no idea how stable it is or whether it even works today): https://github.com/active-expressions/active-expressions-squeak Here is a journal paper about the concept: https://www.hirschfeld.org/writings/media/RamsonHirschfeld_2017_ActiveExpres... I'm not familiar with its implementation, but an efficient implementation might incorporate proxies like ObjectViewer or methods wrappers with source code instrumentation to detect possible changes.
What a great round out for 2023! The quality and quantity of mind-blowing projects from this community is mind-blowing. I'm holding up my glass in (virtual) toast to Squeak for this past and coming year.
Yay! I'm looking forward to what this community achieves in 16r7E8. :-)
- Chris
Best, Christoph
*To get an impression of the speed of the simulator, you can step *through* the following expression, open the window triangle menu of the dialog, uncheck "be modally exclusive", and do whatever you want: self inform: 'while this window is open, the entire ui process is being simulated'.
--- Sent from Squeak Inbox Talk
squeak-dev@lists.squeakfoundation.org
-
Chris Muller -
christoph.thiede＠student.hpi.uni-potsdam.de -
David T Lewis -
Jaromir Matas