A simple Ruby::Box demo

Ruby 4.0.0 has an experimental feature - Ruby Box. It provides in-process isolation for Ruby code execution, by creating a separated spaces within a single Ruby process, preventing code in one box from affecting another or the main application. This article shows a simple use case of the new feature.

Scenario

You are supplying a tool for LLM to execute a piece of Ruby code it generated and return the results of the execution.

The Code

1. The unsafe version

   def unsafe_call(llm_generated_code)
     eval(llm_generated_code)
   end
   

2. The Ruby Box version

   def safe_call(llm_generated_code)
     box = Ruby::Box.new
     box.eval(llm_generated_code)
   end
   

Test the Code

1. Pretend we have a global current_user, like so:

   class User
     attr_accessor :id, :email, :password_hash, :api_key, :role
   
     def initialize(id:, email:, password_hash:, api_key:, role:)
       @id = id
       @email = email
       @password_hash = password_hash
       @api_key = api_key
       @role = role
     end
   end
   
   $current_user = User.new(
     id: 42,
     email: "admin@company.com",
     password_hash: "$2a$12$K8pQXvZ8...",
     api_key: "sk_live_abc123xyz789secretkey",
     role: "admin"
   )
   

2. Consider the following code provided by LLM:

   llm_malicious_code = <<'RUBY'
     result = 2 + 2
   
     stolen_data = {
       user_id: $current_user.instance_variable_get(:@id),
       email: $current_user.instance_variable_get(:@email),
       password_hash: $current_user.instance_variable_get(:@password_hash),
       api_key: $current_user.instance_variable_get(:@api_key),
       role: $current_user.instance_variable_get(:@role)
     }
   
     stolen_data.each { |k, v| puts "   #{k}: #{v}" }
   
     result
   RUBY
   

3. Run the test

   puts "unsafe_call"
   unsafe_call(llm_malicious_code)
   
   puts "safe_call"
   safe_call(llm_malicious_code)
   

4. And we can observe the result - The unsafe version has a credential leaked.

   ruby: warning: Ruby::Box is experimental, and the behavior may change in the future!
   See https://docs.ruby-lang.org/en/4.0/Ruby/Box.html for known issues, etc.
   unsafe_call
      user_id: 42
      email: admin@company.com
      password_hash: $2a$12$K8pQXvZ8...
      api_key: sk_live_abc123xyz789secretkey
      role: admin
   safe_call
      user_id: 
      email: 
      password_hash: 
      api_key: 
      role:
   

Notes

1. The Ruby box demonstrates a strong potential in many use cases, including potentially a safer LLM function-calling environment. Although additional security measures are still required, such as providing a whitelist to "require" and detecting malicious patterns in code.

2. Currently, the feature is not recommended to be used in production, as it has several known issues and bugs.