Skip to content

Segfault in connect under multithreaded use: cTinyTdsError/mTinyTds not registered with the Ruby GC #608

Description

@znppaed7

Environment

  • Ruby 3.4.10 (arm64-darwin, Apple Silicon)
  • tiny_tds 3.4.0 (also reproducible on 3.2.1 and 3.3.0)
  • FreeTDS 1.5.19 (Homebrew)
  • Web server: Falcon / async-http, --threaded (12 native threads)

Summary

Under a multithreaded Ruby server, TinyTds::Client.new intermittently crashes
the whole process with a segmentation fault during connect. The crash happens
while FreeTDS delivers its normal info messages (e.g. "Changed language
setting"), inside tinytds_msg_handler. In a single-threaded context (plain
ruby, or Falcon --forked) the same code runs fine.

Steps to reproduce

  1. Run a multithreaded Ruby HTTP server (e.g. falcon serve --threaded) that
    opens ESB/MS SQL connections per request via TinyTds::Client.new(host:, ...).
  2. Make requests that trigger connect.
  3. The process aborts with [BUG] Segmentation fault.

It does not reproduce in single-threaded execution — the same connect works
in isolated ruby -e 'TinyTds::Client.new(...)' and when the server runs with a
single thread/process.

Key backtrace (from the crash log)

libsybdb.5.dylib(_dblib_handle_info_message)
libsybdb.5.dylib(tds_connect)
libsybdb.5.dylib(tds_connect_and_login)
libsybdb.5.dylib(tdsdbopen)
tiny_tds.bundle(rb_tinytds_connect)
...
tiny_tds.bundle(tinytds_msg_handler)
tiny_tds.bundle(rb_tinytds_raise_error)
libruby(rb_exc_new_cstr)
libruby(rb_class_superclass -> rb_unexpected_type -> unexpected_type)
libruby(rb_obj_as_string -> rb_funcall -> callable_method_entry_or_negative) ← SIGSEGV

The value in x0 at the crash is a fragment of an ASCII string (e.g.
"read \"i\""), i.e. the C code is dereferencing a stale/freed pointer.

Root cause

ext/tiny_tds/tiny_tds_ext.c keeps global VALUEs that are not registered
with the GC
:

VALUE mTinyTds, cTinyTdsError;

void Init_tiny_tds() {
  mTinyTds      = rb_define_module("TinyTds");
  cTinyTdsError = rb_const_get(mTinyTds, rb_intern("Error"));
  init_tinytds_client();
  init_tinytds_result();
}

rb_tinytds_raise_error builds an exception on every FreeTDS message:

e = rb_exc_new2(cTinyTdsError, error.error);

Because cTinyTdsError/mTinyTds are not pinned via
rb_gc_register_address/rb_global_variable (unlike opt_escape_regex and
opt_escape_dblquote in client.c), the GC may move/reclaim the TinyTds
module and TinyTds::Error class while another native thread is mid-connect.
The C pointer then goes stale, and rb_exc_new2 crashes.

Proposed fix

void Init_tiny_tds() {
  mTinyTds      = rb_define_module("TinyTds");
  cTinyTdsError = rb_const_get(mTinyTds, rb_intern("Error"));

  rb_gc_register_address(&mTinyTds);
  rb_gc_register_address(&cTinyTdsError);

  init_tinytds_client();
  init_tinytds_result();
}

Related

Possibly the same family of multithreading crashes as #539 (also unregistered
GC globals / native code + concurrent threads).

Activity

  1. CoderJoshDK commented on Sep 29, 2026

    @CoderJoshDK

    Confirming a similar experience. However, my minimal repo doesn't require multiple threads or really much at all:

    require "tiny_tds"
    
    STDOUT.sync = true
    
    GC.verify_compaction_references(double_heap: true, toward: :empty)
    
    TinyTds::Client.new(
      host: "127.0.0.1",
      port: 1,
      username: "unused",
      password: "unused",
      login_timeout: 1,
    )

    Running this gives you a segfault

    Details

    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/lib/ruby/gems/3.4.0/gems/tiny_tds-3.4.0/lib/tiny_tds/client.rb:57: [BUG] Segmentation fault at 0x0000000000000000
    ruby 3.4.11 (2026-09-23 revision 592f1ffdb3) +PRISM [arm64-darwin23]
    
    -- Crash Report log information --------------------------------------------
       See Crash Report log file in one of the following locations:
         * ~/Library/Logs/DiagnosticReports
         * /Library/Logs/DiagnosticReports
       for more details.
    Don't forget to include the above Crash Report log file in bug reports.
    
    -- Control frame information -----------------------------------------------
    c:0005 p:---- s:0022 e:000021 CFUNC  :connect
    c:0004 p:0367 s:0017 e:000016 METHOD /Users/joshie/.local/share/mise/installs/ruby/3.4.11/lib/ruby/gems/3.4.0/gems/tiny_tds-3.4.0/lib/tiny_tds/client.rb:57 [FINISH]
    c:0003 p:---- s:0011 e:000010 CFUNC  :new
    c:0002 p:0034 s:0006 e:000005 EVAL   test.rb:9 [FINISH]
    c:0001 p:0000 s:0003 E:001d60 DUMMY  [FINISH]
    
    -- Ruby level backtrace information ----------------------------------------
    test.rb:9:in '<main>'
    test.rb:9:in 'new'
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/lib/ruby/gems/3.4.0/gems/tiny_tds-3.4.0/lib/tiny_tds/client.rb:57:in 'initialize'
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/lib/ruby/gems/3.4.0/gems/tiny_tds-3.4.0/lib/tiny_tds/client.rb:57:in 'connect'
    
    -- Threading information ---------------------------------------------------
    Total ractor count: 1
    Ruby thread count for this ractor: 1
    
    -- Machine register context ------------------------------------------------
      x0: 0x0000000000000000  x1: 0x0000000000000d21  x2: 0x000000016cf3ac80
      x3: 0x0000000000000000  x4: 0x0000007a9b17b1c0  x5: 0x0000000000000001
      x6: 0x0000000000000002  x7: 0xffffffffb00007ff x18: 0x0000000000000000
     x19: 0x0000000123348540 x20: 0x0000000000000d21 x21: 0x0000000000000000
     x22: 0x0000000000000000 x23: 0x0000000123348540 x24: 0x00000001033cc000
     x25: 0x00000001033cc000 x26: 0x0000000000000000 x27: 0x000000000001b01a
     x28: 0x0000000000000000  lr: 0x000000010310f698  fp: 0x000000016cf3acc0
      sp: 0x000000016cf3ac70
    
    -- C level backtrace information -------------------------------------------
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_vm_bugreport+0xb6c) [0x103132da8]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_bug_for_fatal_signal+0x100) [0x102f68dc0]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(sigsegv+0x84) [0x1030938b4]
    /usr/lib/system/libsystem_platform.dylib(_sigtramp+0x38) [0x19de50144]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(callable_method_entry_or_negative+0xc0) [0x10310f698]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_vm_search_method_slowpath+0xd4) [0x1031029f4]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(gccct_method_search_slowpath+0x24) [0x10311e004]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_funcallv_scope+0x17c) [0x103113ec8]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_funcall+0x88) [0x103114338]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_obj_as_string+0x44) [0x1030a5874]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(ruby__sfvextra+0xe0) [0x1030999c0]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(BSD_vfprintf+0x60c) [0x103097ad8]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(ruby_vsprintf0+0xa4) [0x1030971d8]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_sprintf+0x58) [0x103097330]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(unexpected_type+0x68) [0x10328f86c]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_unexpected_type+0x30) [0x10328f8b8]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_class_superclass+0x0) [0x102ffec28]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_exc_new_cstr+0x50) [0x102f697c0]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/lib/ruby/gems/3.4.0/gems/tiny_tds-3.4.0/lib/tiny_tds/tiny_tds.bundle(rb_tinytds_raise_error+0x74) [0x12335073c]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/lib/ruby/gems/3.4.0/gems/tiny_tds-3.4.0/lib/tiny_tds/tiny_tds.bundle(tinytds_err_handler+0x1ec) [0x123350a4c]
    /opt/homebrew/Cellar/freetds/1.5.19/lib/libsybdb.5.dylib(dbperror+0x1e8) [0x12344e54c]
    /opt/homebrew/Cellar/freetds/1.5.19/lib/libsybdb.5.dylib(_dblib_handle_err_message) [0x123459818]
    /opt/homebrew/Cellar/freetds/1.5.19/lib/libsybdb.5.dylib(tdserror) [0x1234668f8]
    /opt/homebrew/Cellar/freetds/1.5.19/lib/libsybdb.5.dylib(tds_connect) [0x123467b1c]
    /opt/homebrew/Cellar/freetds/1.5.19/lib/libsybdb.5.dylib(tds_connect_and_login) [0x123466c78]
    /opt/homebrew/Cellar/freetds/1.5.19/lib/libsybdb.5.dylib(tdsdbopen) [0x12344f500]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/lib/ruby/gems/3.4.0/gems/tiny_tds-3.4.0/lib/tiny_tds/tiny_tds.bundle(rb_tinytds_connect+0x30c) [0x123351af4]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x1031248e8]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(vm_exec_core+0x2470) [0x1031087a0]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_vm_exec+0x1e8) [0x103104df0]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_call0+0x3c8) [0x10312b248]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_class_new_instance_pass_kw+0x40) [0x102ffeb30]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x1031248e8]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(vm_exec_core+0x2470) [0x1031087a0]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_vm_exec+0x1e8) [0x103104df0]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(rb_ec_exec_node+0x9c) [0x102f7347c]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(ruby_run_node+0x44) [0x102f73398]
    /Users/joshie/.local/share/mise/installs/ruby/3.4.11/bin/ruby(main+0x68) [0x102ec1c48]
    

    My example enters through tinytds_err_handler, while the original issue enters through tinytds_msg_handler. Both handlers call rb_tinytds_raise_error, which dereferences the same global cTinyTdsError value in rb_exc_new2.

    An investigation found that there are other unregistered class/module globals used after extension initialization: cTinyTdsClient, cTinyTdsResult, cKernel, and cDate. They should be registered or otherwise made compaction-safe.

    These addresses are not registered with Ruby's garbage collector but need to be. I agree with the proposed fix.

  2. plribeiro3000 commented on Sep 30, 2026

    @plribeiro3000
    Contributor

    We hit this in production as well, with a slightly different profile that may help narrow it down.

    Environment

    • Ruby 4.0.6 (x86_64-linux, +YJIT +PRISM), official ruby:4.0.6 Docker image
    • tiny_tds 3.1.0, compiled from source against the distro freetds-dev package
    • Sequel 5.108.0 (tinytds adapter), Sidekiq 8.1.7, Rails 8.1.3.1
    • Server: Azure SQL Database (Microsoft SQL Azure (RTM) - 12.0.2000.8)

    Summary

    A Sidekiq worker process (24 Ruby threads, concurrency 10) crashed with [BUG] Segmentation fault while opening a new connection through Sequel.connect -> TinyTds::Client.new. The server was healthy: the same connection succeeded moments later from a console. It happened once in roughly two months across about a dozen worker processes that open connections daily, so it is rare but fatal — the whole process dies and the job running on it is lost.

    Notably, nothing in our bundle calls GC.compact, GC.auto_compact= or GC.verify_compaction_references, so this was not induced by forced compaction as in the minimal repro above.

    Key backtrace

    Ruby level:

    sequel-5.108.0/lib/sequel/adapters/tinytds.rb:16:in 'connect'
    tiny_tds-3.1.0/lib/tiny_tds/client.rb:60:in 'initialize'
    tiny_tds-3.1.0/lib/tiny_tds/client.rb:60:in 'connect'
    tiny_tds-3.1.0/lib/tiny_tds/client.rb:60: [BUG] Segmentation fault at 0x000000000000003c
    

    C level (top frames):

    libruby.so.4.0(rb_bug_for_fatal_signal+0x106)
    libruby.so.4.0(sigsegv+0x42)
    libruby.so.4.0(RTYPEDDATA_GET_DATA+0x4) ./include/ruby/internal/core/rtypeddata.h:541
    libruby.so.4.0(rb_managed_id_table_lookup) id_table.c:409
    libruby.so.4.0(vm_lookup_cc+0x51) vm_insnhelper.c:2225
    libruby.so.4.0(rb_funcallv) vm_eval.c:1080
    libruby.so.4.0(rb_obj_as_string) string.c:1858
    libruby.so.4.0(rb_sprintf+0x99) sprintf.c:1223
    libruby.so.4.0(unexpected_type+0x55) error.c:1323
    libruby.so.4.0(rb_unexpected_type)
    libruby.so.4.0(Check_Type+0x0) value_type.h:447
    libruby.so.4.0(rb_exc_new+0x35) error.c:1472
    tiny_tds.so(rb_tinytds_raise_error+0x38) ext/tiny_tds/client.c:36
    tiny_tds.so(tinytds_msg_handler+0xbb) ext/tiny_tds/client.c:178
    libsybdb.so.5
    

    Same path as the original report: tinytds_msg_handler -> rb_tinytds_raise_error -> rb_exc_new2(cTinyTdsError, ...). The Check_Type -> unexpected_type frames show Ruby rejecting the class argument itself, i.e. cTinyTdsError no longer pointed to a valid class at the moment of the crash — the segfault then happens while Ruby tries to format that type-error message.

    Additional context

    • In 3.1.0, cTinyTdsError is the only one of these globals obtained with rb_const_get from a class defined in Ruby (lib/tiny_tds/error.rb); mTinyTds, cTinyTdsClient, cTinyTdsResult, cKernel and cDate are also stored without rb_global_variable / rb_gc_register_address, while opt_escape_regex, opt_escape_dblquote and the opt_* values in result.c are registered.
    • The v4 branch (V4 #603) still has the same unregistered cTinyTdsError in tiny_tds_ext.c, so the rewrite does not fix it on its own.

    +1 to the proposed fix of registering these globals with the GC. We are happy to open a PR with it (plus a regression test based on the compaction repro above) if that is welcome.

  3. CoderJoshDK commented on Sep 30, 2026

    @CoderJoshDK

    In production, we have added

    diff --git a/config/initializers/tiny_tds_gc_guard.rb b/config/initializers/tiny_tds_gc_guard.rb
    new file mode 100644
    index 000000000..a9f80b27f
    --- /dev/null
    +++ b/config/initializers/tiny_tds_gc_guard.rb
    @@ -0,0 +1,25 @@
    +# frozen_string_literal: true
    +
    +# TinyTDS 3.4.0 keeps these Ruby objects in C globals that are not registered
    +# with Ruby's GC. Pin their canonical values so compaction cannot leave the
    +# extension with stale pointers. Remove this after upstream issue #608 is fixed.
    +# https://github.com/rails-sqlserver/tiny_tds/issues/608
    +require "fiddle"
    +require "tiny_tds"
    +
    +register_mark_object = Fiddle::Function.new(
    +  Fiddle::Handle::DEFAULT["rb_gc_register_mark_object"],
    +  [Fiddle::TYPE_VOIDP],
    +  Fiddle::TYPE_VOID,
    +)
    +
    +[
    +  TinyTds,
    +  TinyTds::Error,
    +  TinyTds::Client,
    +  TinyTds::Result,
    +  Kernel,
    +  Date,
    +].each do |object|
    +  register_mark_object.call(Fiddle.dlwrap(object))
    +end

    So far, the segfault has gone away. This is a workaround. And also, we don't know for 100% certainty that this fixes everything; since the crash was semi rare. That said, we were seeing it happen maybe 3 times a day. And it has not happened at all since this code was deployed. So presumably it worked.

    And yes, the minimal repo I provided is not the way we get the crash in production. It was just my way of forcing the same type of crash in a super reliable way.

  4. added 2 commits that reference this issue on Sep 30, 2026
    2fa810b
    df658bc
  5. added a commit that references this issue on Oct 5, 2026
    74a2232
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions