[pull] master from ruby:master

NEWS.md

@@ -12,7 +12,7 @@ Note that each entry is kept to a minimum, see links for details.
 
 * Logical binary operators (`||`, `&&`, `and` and `or`) at the
   beginning of a line continue the previous line, like fluent dot.
-  The following two code are equal:
+  The following two code examples are equal:
 
     ```ruby
     if condition1
@@ -165,7 +165,7 @@ Note: We're only listing outstanding class updates.
         * `Ractor::Port#close`
         * `Ractor::Port#closed?`
 
-        As result, `Ractor.yield` and `Ractor#take` were removed.
+        As a result, `Ractor.yield` and `Ractor#take` were removed.
 
     * `Ractor#join` and `Ractor#value` were added to wait for the
       termination of a Ractor. These are similar to `Thread#join`
@@ -182,7 +182,7 @@ Note: We're only listing outstanding class updates.
 
     * `Ractor#close_incoming` and `Ractor#close_outgoing` were removed.
 
-    * `Ractor.shareable_proc` and `Ractor.shareable_lambda` is introduced
+    * `Ractor.shareable_proc` and `Ractor.shareable_lambda` are introduced
       to make shareable Proc or lambda.
       [[Feature #21550]], [[Feature #21557]]
 
@@ -214,7 +214,7 @@ Note: We're only listing outstanding class updates.
     * `Set` is now a core class, instead of an autoloaded stdlib class.
       [[Feature #21216]]
 
-    * `Set#inspect` now uses a simpler displays, similar to literal arrays.
+    * `Set#inspect` now uses a simpler display, similar to literal arrays.
       (e.g., `Set[1, 2, 3]` instead of `#<Set: {1, 2, 3}>`). [[Feature #21389]]
 
     * Passing arguments to `Set#to_set` and `Enumerable#to_set` is now deprecated.
@@ -358,7 +358,7 @@ The following bundled gems are updated.
     * `Ractor.yield`
     * `Ractor#take`
     * `Ractor#close_incoming`
-    * `Ractor#close_outgoging`
+    * `Ractor#close_outgoing`
 
     [[Feature #21262]]
 
@@ -369,7 +369,7 @@ The following bundled gems are updated.
 
 * `rb_path_check` has been removed. This function was used for
   `$SAFE` path checking which was removed in Ruby 2.7,
-  and was already deprecated,.
+  and was already deprecated.
   [[Feature #20971]]
 
 * A backtrace for `ArgumentError` of "wrong number of arguments" now
@@ -449,22 +449,36 @@ The following bundled gems are updated.
 
 ## Implementation improvements
 
+* `Class#new` (ex. `Object.new`) is faster in all cases, but especially when passing keyword arguments. This has also been integrated into YJIT and ZJIT. [[Feature #21254]]
+* GC heaps of different size pools now grow independently, reducing memory usage when only some pools contain long-lived objects
+* GC sweeping is faster on pages of large objects
+* "Generic ivar" objects (String, Array, `TypedData`, etc.) now use a new internal "fields" object for faster instance variable access
+* The GC avoids maintaining an internal `id2ref` table until it is first used, making `object_id` allocation and GC sweeping faster
+* `object_id` and `hash` are faster on Class and Module objects
+* Larger bignum Integers can remain embedded using variable width allocation
+* `Random`, `Enumerator::Product`, `Enumerator::Chain`, `Addrinfo`,
+  `StringScanner`, and some internal objects are now write-barrier protected,
+  which reduces GC overhead.
+
 ### Ractor
 
-A lot of work has gone into making Ractors more stable, performant, and usable. These improvements bring Ractors implementation closer to leaving experimental status.
+A lot of work has gone into making Ractors more stable, performant, and usable. These improvements bring Ractor implementation closer to leaving experimental status.
 
 * Performance improvements
-    * Frozen strings and the symbol table internally use a lock-free hash set
+    * Frozen strings and the symbol table internally use a lock-free hash set [[Feature #21268]]
     * Method cache lookups avoid locking in most cases
-    * Class (and geniv) instance variable access is faster and avoids locking
-    * Cache contention is avoided during object allocation
+    * Class (and generic ivar) instance variable access is faster and avoids locking
+    * CPU cache contention is avoided in object allocation by using a per-ractor counter
+    * CPU cache contention is avoided in xmalloc/xfree by using a thread-local counter
     * `object_id` avoids locking in most cases
 * Bug fixes and stability
     * Fixed possible deadlocks when combining Ractors and Threads
     * Fixed issues with require and autoload in a Ractor
     * Fixed encoding/transcoding issues across Ractors
     * Fixed race conditions in GC operations and method invalidation
     * Fixed issues with processes forking after starting a Ractor
+    * GC allocation counts are now accurate under Ractors
+    * Fixed TracePoints not working after GC [[Bug #19112]]
 
 ## JIT
 
@@ -488,6 +502,7 @@ A lot of work has gone into making Ractors more stable, performant, and usable.
 [Feature #15408]: https://bugs.ruby-lang.org/issues/15408
 [Feature #17473]: https://bugs.ruby-lang.org/issues/17473
 [Feature #18455]: https://bugs.ruby-lang.org/issues/18455
+[Bug #19112]:     https://bugs.ruby-lang.org/issues/19112
 [Feature #19630]: https://bugs.ruby-lang.org/issues/19630
 [Bug #19868]:     https://bugs.ruby-lang.org/issues/19868
 [Feature #19908]: https://bugs.ruby-lang.org/issues/19908
@@ -510,6 +525,7 @@ A lot of work has gone into making Ractors more stable, performant, and usable.
 [Feature #21219]: https://bugs.ruby-lang.org/issues/21219
 [Feature #21254]: https://bugs.ruby-lang.org/issues/21254
 [Feature #21258]: https://bugs.ruby-lang.org/issues/21258
+[Feature #21268]: https://bugs.ruby-lang.org/issues/21268
 [Feature #21262]: https://bugs.ruby-lang.org/issues/21262
 [Feature #21275]: https://bugs.ruby-lang.org/issues/21275
 [Feature #21287]: https://bugs.ruby-lang.org/issues/21287
@@ -523,6 +539,7 @@ A lot of work has gone into making Ractors more stable, performant, and usable.
 [Feature #21527]: https://bugs.ruby-lang.org/issues/21527
 [Feature #21543]: https://bugs.ruby-lang.org/issues/21543
 [Feature #21550]: https://bugs.ruby-lang.org/issues/21550
+[Feature #21552]: https://bugs.ruby-lang.org/issues/21552
 [Feature #21557]: https://bugs.ruby-lang.org/issues/21557
 [Bug #21654]:     https://bugs.ruby-lang.org/issues/21654
 [Feature #21678]: https://bugs.ruby-lang.org/issues/21678

ast.c

@@ -32,9 +32,13 @@ static size_t
 node_memsize(const void *ptr)
 {
     struct ASTNodeData *data = (struct ASTNodeData *)ptr;
-    rb_ast_t *ast = rb_ruby_ast_data_get(data->ast_value);
+    size_t size = sizeof(struct ASTNodeData);
+    if (data->ast_value) {
+        rb_ast_t *ast = rb_ruby_ast_data_get(data->ast_value);
+        size += rb_ast_memsize(ast);
+    }
 
-    return sizeof(struct ASTNodeData) + rb_ast_memsize(ast);
+    return size;
 }
 
 static const rb_data_type_t rb_node_type = {

bootstraptest/test_ractor.rb

@@ -316,7 +316,7 @@ def test n
 } unless (ENV.key?('TRAVIS') && ENV['TRAVIS_CPU_ARCH'] == 'arm64') # https://bugs.ruby-lang.org/issues/17878
 
 # Exception for empty select
-assert_match /specify at least one ractor/, %q{
+assert_match /specify at least one Ractor::Port or Ractor/, %q{
   begin
     Ractor.select
   rescue ArgumentError => e

box.c

@@ -276,10 +276,15 @@ box_entry_free(void *ptr)
 static size_t
 box_entry_memsize(const void *ptr)
 {
+    size_t size = sizeof(rb_box_t);
     const rb_box_t *box = (const rb_box_t *)ptr;
-    return sizeof(rb_box_t) + \
-        rb_st_memsize(box->loaded_features_index) + \
-        rb_st_memsize(box->loading_table);
+    if (box->loaded_features_index) {
+        size += rb_st_memsize(box->loaded_features_index);
+    }
+    if (box->loading_table) {
+        size += rb_st_memsize(box->loading_table);
+    }
+    return size;
 }
 
 static const rb_data_type_t rb_box_data_type = {

compar.c

@@ -123,10 +123,14 @@ cmp_ge(VALUE x, VALUE y)
 
 /*
  *  call-seq:
- *     obj < other    -> true or false
+ *    self < other -> true or false
+ *
+ *  Returns whether +self+ is "less than" +other+;
+ *  equivalent to <tt>(self <=> other) < 0</tt>:
+ *
+ *    'foo' < 'foo'  # => false
+ *    'foo' < 'food' # => true
  *
- *  Compares two objects based on the receiver's <code><=></code>
- *  method, returning true if it returns a value less than 0.
  */
 
 static VALUE

doc/language/packed_data.rdoc

@@ -53,6 +53,8 @@ These tables summarize the directives for packing and unpacking.
 
   U             | UTF-8 character
   w             | BER-compressed integer
+  R             | LEB128 encoded unsigned integer
+  r             | LEB128 encoded signed integer
 
 === For Floats
 

doc/string/aset.rdoc

@@ -170,9 +170,9 @@ With string argument +substring+ given:
   s['ll'] = 'foo'  # => "foo"
   s  # => "hefooo"
 
-  s = 'тест'
-  s['ес'] = 'foo'  # => "foo"
-  s  # => "тfooт"
+  s = 'Привет'
+  s['ив'] = 'foo'  # => "foo"
+  s  # => "Прfooет"
 
   s = 'こんにちは'
   s['んにち'] = 'foo'  # => "foo"

doc/string/bytes.rdoc

@@ -1,7 +1,7 @@
 Returns an array of the bytes in +self+:
 
-  'hello'.bytes # => [104, 101, 108, 108, 111]
-  'тест'.bytes  # => [209, 130, 208, 181, 209, 129, 209, 130]
+  'hello'.bytes  # => [104, 101, 108, 108, 111]
+  'Привет'.bytes # => [208, 159, 209, 128, 208, 184, 208, 178, 208, 181, 209, 130]
   'こんにちは'.bytes
   # => [227, 129, 147, 227, 130, 147, 227, 129, 171, 227, 129, 161, 227, 129, 175]
 

doc/string/bytesize.rdoc

@@ -5,9 +5,9 @@ Note that the byte count may be different from the character count (returned by
   s = 'foo'
   s.bytesize # => 3
   s.size     # => 3
-  s = 'тест'
-  s.bytesize # => 8
-  s.size     # => 4
+  s = 'Привет'
+  s.bytesize # => 12
+  s.size     # => 6
   s = 'こんにちは'
   s.bytesize # => 15
   s.size     # => 5

doc/string/center.rdoc

@@ -9,7 +9,7 @@ centered and padded on one or both ends with +pad_string+:
   'hello'.center(20, '-|')      # => "-|-|-|-hello-|-|-|-|" # Some padding repeated.
   'hello'.center(10, 'abcdefg') # => "abhelloabc"           # Some padding not used.
   '  hello  '.center(13)        # => "    hello    "
-  'тест'.center(10)             # => "   тест   "
+  'Привет'.center(10)           # => "  Привет  "
   'こんにちは'.center(10)         # => "  こんにちは   "      # Multi-byte characters.
 
 If +size+ is less than or equal to the size of +self+, returns an unpadded copy of +self+:

doc/string/chars.rdoc

@@ -1,7 +1,7 @@
 Returns an array of the characters in +self+:
 
   'hello'.chars     # => ["h", "e", "l", "l", "o"]
-  'тест'.chars      # => ["т", "е", "с", "т"]
+  'Привет'.chars    # => ["П", "р", "и", "в", "е", "т"]
   'こんにちは'.chars # => ["こ", "ん", "に", "ち", "は"]
   ''.chars          # => []
 

hash.c

@@ -4915,10 +4915,9 @@ rb_hash_le(VALUE hash, VALUE other)
 
 /*
  *  call-seq:
- *    self < other_hash -> true or false
+ *    self < other -> true or false
  *
- *  Returns +true+ if the entries of +self+ are a proper subset of the entries of +other_hash+,
- *  +false+ otherwise:
+ *  Returns whether the entries of +self+ are a proper subset of the entries of +other+:
  *
  *    h = {foo: 0, bar: 1}
  *    h < {foo: 0, bar: 1, baz: 2} # => true   # Proper subset.

id_table.c

@@ -374,7 +374,7 @@ rb_managed_id_table_create(const rb_data_type_t *type, size_t capa)
     struct rb_id_table *tbl;
     VALUE obj = TypedData_Make_Struct(0, struct rb_id_table, type, tbl);
     RB_OBJ_SET_SHAREABLE(obj);
-    rb_id_table_init(tbl, capa);
+    rb_id_table_init(tbl, capa); // NOTE: this can cause GC, so dmark and dsize need to check tbl->items
     return obj;
 }
 

include/ruby/internal/core/rtypeddata.h

@@ -615,13 +615,24 @@ RBIMPL_ATTR_ARTIFICIAL()
  * directly.
  */
 static inline void *
-rbimpl_check_typeddata(VALUE obj, const rb_data_type_t *type)
+rbimpl_check_typeddata(VALUE obj, const rb_data_type_t *expected_type)
 {
-    if (RB_LIKELY(RB_TYPE_P(obj, T_DATA) && RTYPEDDATA_P(obj) && RTYPEDDATA_TYPE(obj) == type)) {
-        return RTYPEDDATA_GET_DATA(obj);
+    if (RB_LIKELY(RB_TYPE_P(obj, T_DATA) && RTYPEDDATA_P(obj))) {
+        const rb_data_type_t *actual_type = RTYPEDDATA_TYPE(obj);
+        void *data = RTYPEDDATA_GET_DATA(obj);
+        if (RB_LIKELY(actual_type == expected_type)) {
+            return data;
+        }
+
+        while (actual_type) {
+            actual_type = actual_type->parent;
+            if (actual_type == expected_type) {
+                return data;
+            }
+        }
     }
 
-    return rb_check_typeddata(obj, type);
+    return rb_check_typeddata(obj, expected_type);
 }
 
 

iseq.c

@@ -3936,7 +3936,6 @@ rb_vm_insn_decode(const VALUE encoded)
 static inline int
 encoded_iseq_trace_instrument(VALUE *iseq_encoded_insn, rb_event_flag_t turnon, bool remain_traced)
 {
-    ASSERT_vm_locking();
     st_data_t key = (st_data_t)*iseq_encoded_insn;
     st_data_t val;
 

jit.c

@@ -23,7 +23,14 @@ enum jit_bindgen_constants {
     ROBJECT_OFFSET_AS_ARY = offsetof(struct RObject, as.ary),
 
     // Field offsets for the RString struct
-    RUBY_OFFSET_RSTRING_LEN = offsetof(struct RString, len)
+    RUBY_OFFSET_RSTRING_LEN = offsetof(struct RString, len),
+
+    // Field offsets for rb_execution_context_t
+    RUBY_OFFSET_EC_CFP = offsetof(rb_execution_context_t, cfp),
+    RUBY_OFFSET_EC_INTERRUPT_FLAG = offsetof(rb_execution_context_t, interrupt_flag),
+    RUBY_OFFSET_EC_INTERRUPT_MASK = offsetof(rb_execution_context_t, interrupt_mask),
+    RUBY_OFFSET_EC_THREAD_PTR = offsetof(rb_execution_context_t, thread_ptr),
+    RUBY_OFFSET_EC_RACTOR_ID = offsetof(rb_execution_context_t, ractor_id),
 };
 
 // Manually bound in rust since this is out-of-range of `int`,

numeric.c

@@ -1702,15 +1702,15 @@ flo_ge(VALUE x, VALUE y)
  *  call-seq:
  *    self < other -> true or false
  *
- *  Returns +true+ if +self+ is numerically less than +other+:
+ *  Returns whether the value of +self+ is less than the value of +other+;
+ *  +other+ must be numeric, but may not be Complex:
  *
  *    2.0 < 3              # => true
  *    2.0 < 3.0            # => true
  *    2.0 < Rational(3, 1) # => true
  *    2.0 < 2.0            # => false
  *
  *  <tt>Float::NAN < Float::NAN</tt> returns an implementation-dependent value.
- *
  */
 
 static VALUE
@@ -5038,16 +5038,15 @@ fix_lt(VALUE x, VALUE y)
  * call-seq:
  *    self < other -> true or false
  *
- * Returns +true+ if the value of +self+ is less than that of +other+:
+ * Returns whether the value of +self+ is less than the value of +other+;
+ * +other+ must be numeric, but may not be Complex:
  *
  *    1 < 0              # => false
  *    1 < 1              # => false
  *    1 < 2              # => true
  *    1 < 0.5            # => false
  *    1 < Rational(1, 2) # => false
  *
- *  Raises an exception if the comparison cannot be made.
- *
  */
 
 static VALUE

object.c

@@ -1948,14 +1948,15 @@ rb_class_inherited_p(VALUE mod, VALUE arg)
 
 /*
  * call-seq:
- *   mod < other   ->  true, false, or nil
+ *   self < other -> true, false, or nil
  *
- * Returns true if <i>mod</i> is a subclass of <i>other</i>. Returns
- * <code>false</code> if <i>mod</i> is the same as <i>other</i>
- * or <i>mod</i> is an ancestor of <i>other</i>.
- * Returns <code>nil</code> if there's no relationship between the two.
- * (Think of the relationship in terms of the class definition:
- * "class A < B" implies "A < B".)
+ * Returns whether +self+ is a subclass of +other+,
+ * or +nil+ if there is no relationship between the two:
+ *
+ *   Float < Numeric # => true
+ *   Numeric < Float # => false
+ *   Float < Float   # => false
+ *   Float < Hash    # => nil
  *
  */