There used to be quite a bit of interest in Prolog circles in automatic
cycle detection.  Of course the comparative absence of side effects in
Prolog makes it a *lot* easier.  The major surviving Prolog systems don't
bother, in part because keeping accurate track of stack depth conflicts
with tail recursion optimisation.  The Clean and Haskell compilers I have
access to basically let you set a limit on stack size (if you want to) and
when the hardware/OS tell you it has run out, kill your program.

Interlisp-D on the Xerox Dandelions and such (1108, 1109, 1185, 1186) had a
hard limit of 64k cells in the stack.  It wasn't that hard to run out, but
somehow most Interlisp programmers never noticed that there was a limit.

Preferences are entirely the wrong way to go because they have to be set by
the wrong person, who lacks the knowledge about how the limit might be set.
Something like
    aBlock valueWithStackLimit: aNumber
would let the programmer override the limit at the point where s/he has some
idea what limit will be needed.

Even that's a bit dodgy.  The main thing is that it's ok to blast away the
thread that gets too big, but not to crash the entire Smalltalk system.