Hi Jaromir,
In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
The TraceDebugger now honors the suspended/terminated state of the debugged process so you cannot step beyond Processor activeProcess suspend any longer, like in a regular debugger.
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2024-01-02T19:53:31+01:00, christoph.thiede@student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me. But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: Context runSimulated: [Context runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:, #terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate Simulator debug: [thisContext findNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContext findNextHandlerContextStarting] debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContext class] debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulator debug: [thisContext findNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Example: do step through to the [^2] block and then step through again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to the traditional one?
In other words: if I run the traditional debugger and the Trace one side by side, should they display analogous steps?
In the other message I wrote about a different way to simulate primitive calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn: context as if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net> wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would be the best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would help tremendously to be able to ask right away instead of trudging through the code/help :) The code usually helps to understand **how** things work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new functionalities "step-by-step", and not be overwhelmed by all the might of the call tree :) Being able to go back is already a hell of an improvement! Or when you open an inspector on a context the context's state gets frozen in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the primitive call. The Trace debugger, however, starts executing the fallback code of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me. :-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is not ready for all your clever investigations regarding non-local returns and unwinding. ;-) In fact, your example reveals another limitation that I forgot to mention in the announcement, which regards programs with irregular context switches - e.g., generators/coroutines, but also non-local returns through unwind contexts. This is because the TraceDebugger stores and displays all method invocations in a tree, but in the case of manual context switches, there is no single global tree - its structure would change over the execution time, and when selecting a method invocation, it is not even clear to what parent (sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack at the viewed point in time (see also the '@ <timeIndex>' in the window title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You would notice the same in a normal debugger if you turned off the optional primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to step out of #terminateTo: again. (You can update the TraceDebugger from the window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step further and eventually return back to the starting point. :-) If you want to, you can also turn off the preference "Show call tree in TraceDebugger" to make the TraceDebugger look more like a normal debugger, which also solves the context switches issue. But in general - unless you are debugging unwinding stuff - I would not recommend that as it removes one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the TraceDebugger. Better use it to explore how the simulator works. :-) For example, you could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method, press Cmd + f(ind), and type "return:from:" to investigate the behavior of your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort some things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
This indeed sounds like a GREAT idea! I look forward to seeing your
use
cases to build the right intuition.
In the meantime I've tried to debug/trace this example I've been
working
with lately:
[^2] ensure: []
If I start the debugger, hit `trace it` and then `step over`, it
stops
at Context>>terminate and the view gets corrupted (the initial part
of
the trace is hidden and can't be made visible unless clicking on
some of
the pink lines - but not every line does it...)
If I then continue stepping over it ends up with some kind of error:
Maybe this is just an unfortunate example... Or maybe I'm just doing something wrong...
At any rate - THANKS for your effort!!
On 30-Dec-23 4:37:28 PM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Thanks for the reply, Dave! I will try to post one or two concrete
use
cases about the TraceDebugger in the next couple of days, so stay tuned. :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote:
> This sounds like really interesting work! I love the idea of
being
> able to interactively go back in "oops, I‘ve stepped too far,
let‘s
> start all over again" situations. It will probably take some
time for
> me and others to wrap our heads around the things you have done,
so
> don't be surprised if you get a delayed response to this
announcement
> :-) > > Congratulations! > Dave > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > Hi all! > > > > I‘m very excited to announce a project today that we have been > > working on over the past two years: The *TraceDebugger* [1] is
a
new > > back-in-time/time-travel/omniscient debugging tool for Squeak
that
> > allows you to record past method activations and states during > > execution and explore them later. > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > Metacellonew > > baseline:'TraceDebugger'; > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > /"repository: > > 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > 6.0"/ > > get; > > load. > > > > **What can it do? (Features)** > > > > - *Record all method activations and historic states:*
Normally
step > > through a program in the debugger while automatically
recording its
> > execution. > > - *Replay execution of a traced program:* Navigate through all > > method invocations using the /*context tree*/ or the /*Step Back/Step > > Forward*/ buttons (to avoid these "oops, I‘ve stepped too far, > > let‘s start all over again" situations). > > - *Interact with historic states:* Inspect/explore snapshots
of
> > objects or send them any message. > > - *State-centric debugging using the ***/History
Explorer*/**:*
> > Gather, explore, and visualize all changes to an
object/expression
> > over the recorded time ("When did this variable/collection/screenshot > > change?"). > > - *Additional navigation tools* for searching and filtering
the
> > context tree. > > - *Focus on interactivity:* No hours of recording, no GBs of
mem
> > consumption - at least for common small to medium programs. > > - *UI resembles the classic Smalltalk debugger:* You'll find
your
> > familiar stepping buttons, code browsing tools, inspectors,
and
> > shortcuts - plus more. > > > > The TraceDebugger is a general-purpose tool and not tied to > > particular domains. In the past months, we have successfully
used
it > > to understand several bugs and interaction patterns in the
Trunk
> > (Morphic layout/rendering, compiler/decompiler, code
simulation,
> > …). The tool is also self-supporting, so you can debug a > > TraceDebugger from another TraceDebugger. :-) > > > > **What can‘t it do (yet)? (Limitations and future work)** > > > > - *High performance:* While (sufficiently) fast enough for
most
> > small to medium workloads, tracing very compute- or
mem-intensive
> > operations may require more time (ex.: compiler/decompiler > > invocation: <1s, HTTPS request: <10s, tool building: <5m,
complex
> > rendering: minutes up to hours). > > - *Not a dataflow analyzer:* The TraceDebugger does not track > > dataflow events (e.g., argument passing) but only state
changes.
> > - *No tracing of external states/events* for FFI/OSProcess or custom > > VM modules. > > - *No support for advanced language concepts* such as identity > > forwarding/write barriers. > > > > **How does it work? (Implementation)** > > > > In one sentence: To record message sends and side effects, we > > decorate the execution of certain bytecodes with tracing
extensions
> > by modifying the code simulation using SimulationStudio [2]. > > > > In one paragraph: The program is executed in a specialized
code
> > simulator that overrides instructions for sending messages
(e.g.,
> > send, superSend) and for performing side-effects (e.g., popIntoRcvr, > > primitiveAtPut, push). All message sends are recorded in a
tree and
> > all changed object slots are stored in a sparse time-dependent memory > > structure before they are overwritten. For time-traveling, the
tree
> > is traversed using a cursor. For accessing historic objects, a proxy > > evaluates all messages sent to an object in another
specialized
> > simulator (retracing simulator) that emulates historic states
for
the > > requested point in time by forwarding read primitives (e.g., > > pushRcvr, primitiveAt) to the recorded memory. For gathering
state
> > changes in the History Explorer efficiently, the query is
evaluated
> > in a range retracing simulator with vectorization and fork semantics. > > > > In academic terms: We have published two papers about the > > TraceDebugger that provide further details about its
implementation
> > and its applications for program exploration, "Object-Centric > > Time-Travel Debugging: Exploring Traces of Objects" [3] and > > "Time-Awareness in Object Exploration Tools: Toward In Situ > > Omniscient Debugging" [4]. > > > > In Smalltalk: Just check out the code base and explore it by > > yourself! The class comments in TraceDebugger code://TraceDebugger > > and TDBCursor code://TDBCursor should provide good starting points. > > > > **How can I use it?** > > > > Please try it out and report feedback! The TraceDebugger
supports
> > the latest Squeak Trunk and Squeak 6.0. You can either
download a
> > prepared all-in-one bundle on GitHub: > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger/releases > > > > Or you can install it into your own image using Metacello: > > > > Metacellonew > > baseline:'TraceDebugger'; > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > /"repository: > > 'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak
> > 6.0"/ > > get; > > load. > > > > To get started, just open a normal debugger (e.g., by
selecting an
> > expression and pressing Cmd+Shift+D to debug it) and then
press the
> > "Trace It" button on the right. There‘s also some pretty
detailed
> > documentation in the Help Browser <code:// TraceDebugger
showHelp>
> > that covers everything you should know. > > > > My goal is to improve convenience and provide a useful tool
for the
> > community, so I‘m very excited to hear your impressions,
ideas, and
> > thoughts. Here, on GitHub, or in a private message. Let‘s have
a
> > great discussion! :-) > > > > Best, > > > > Christoph (and Marcel) > > > > PS: Props to Eliot who brought up the original idea of
"subclassing
> > from Context" for other reasons four years ago. [5] > > > > [1] https://github.com/hpi-swa-lab/squeak-tracedebugger > > [2] https://github.com/LinqLover/SimulationStudio > > [3] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
> > Object-Centric Time-Travel Debugging: Exploring Traces of
Objects.
> > https://doi.org/10.1145/3594671.3594678 In /Companion
Proceedings
> > of the 7th International Conference on the Art, Science, and > > Engineering of Programming/ (/<Programming>'23 Companion/),
March
> > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7 pages.
DOI:
> > 10.1145/3594671.3594678
https://doi.org/10.1145/3594671.3594678.
> > PDF: https://dl.acm.org/doi/pdf/10.1145/3594671.3594678 > > [4] Christoph Thiede, Marcel Taeumel, and Robert Hirschfeld.
> > Time-Awareness in Object Exploration Tools: Toward In Situ Omniscient > > Debugging. https://dl.acm.org/doi/10.1145/3622758.3622892 In > > /Proceedings of the 2023 ACM SIGPLAN International Symposium
on New
> > Ideas, New Paradigms, and Reflections on Programming and
Software/
> > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal. ACM,
New
> > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > https://doi.org/10.1145/3622758.3622892. PDF: > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > [5] > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > --- > > /Sent from//Squeak Inbox Talk > > https://github.com/hpi-swa-lab/squeak-inbox-talk/