[Rust] array::transform::tests::test_struct failed #18457

Activity

1. asfimport commented on Jan 15, 2021

   

   asfimport

   on Jan 15, 2021

   Author

   More actions

   Qingyou Meng / @mqy:
   @jorgecarleitao

   Test passed if construct the expect directly (not from slice()).

   look like the problem is caused by ArrayRef::slice(), which contains cloned buffers and null bitmap.

   1. Add new array_data_slice_problem to verify failure case 1:

   
       #[test]
       fn array_bug_1() {
           let strings: ArrayRef = Arc::new(StringArray::from(vec![
               None,
               Some("b"),
           ]));
           let strings_slice = strings.slice(1, 1);
           let strings2: ArrayRef = Arc::new(StringArray::from(vec![
               Some("b"),
           ]));
           assert_eq!(&strings2, &strings_slice, "\n##1: {:?}\n##2: {:?}", strings2.data(), strings_slice.data());
       }
   
   Failed with error:
   
   thread 'array::transform::tests::array_bug_1' panicked at 'assertion failed: `(left == right)`
     left: `StringArray
   [
     "b",
   ]`,
    right: `StringArray
   [
     "b",
   ]`: 
   ##1: ArrayData { data_type: Utf8, len: 1, null_count: 0, offset: 0, buffers: [Buffer { data: Bytes { ptr: 0x7fd413704600, len: 8, data: [0, 0, 0, 0, 1, 0, 0, 0] }, offset: 0 }, Buffer { data: Bytes { ptr: 0x7fd413704700, len: 1, data: [98] }, offset: 0 }], child_data: [], null_bitmap: Some(Bitmap { bits: Buffer { data: Bytes { ptr: 0x7fd413704680, len: 1, data: [1] }, offset: 0 } }) }
   ##2: ArrayData { data_type: Utf8, len: 1, null_count: 0, offset: 1, buffers: [Buffer { data: Bytes { ptr: 0x7fd413704300, len: 12, data: [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0] }, offset: 0 }, Buffer { data: Bytes { ptr: 0x7fd413704400, len: 1, data: [98] }, offset: 0 }], child_data: [], null_bitmap: Some(Bitmap { bits: Buffer { data: Bytes { ptr: 0x7fd413704380, len: 1, data: [2] }, offset: 0 } }) }', arrow/src/array/transform/mod.rs:823:9
   

    

   2. Adjust array_data_slice_problem, add element Some("a"):

    

   
       #[test]
       fn array_bug_2() {
           let strings: ArrayRef = Arc::new(StringArray::from(vec![
               Some("a"),
               None,
               Some("b"),
           ]));
           let strings_slice = strings.slice(2, 1);
           let strings2: ArrayRef = Arc::new(StringArray::from(vec![
               Some("b"),
           ]));
           assert_eq!(&strings2, &strings_slice, "\n##1: {:?}\n##2: {:?}", strings2.data(), strings_slice.data());
       }
   
   
   Panic on call to `strings.slice(2, 1)`:
   
   thread 'array::transform::tests::array_bug_2' panicked at 'range end index 2 out of range for slice of length 0', arrow/src/array/equal/utils.rs:81:49
   
   

    
2. asfimport commented on Jan 15, 2021

   

   asfimport

   on Jan 15, 2021

   Author

   More actions

   Jorge Leitão / @jorgecarleitao:
   great finding. cc [~nevi_me]
3. asfimport commented on Jan 15, 2021

   

   asfimport

   on Jan 15, 2021

   Author

   More actions

   Andrew Lamb / @alamb:
   Thanks @mqy – this is a really nice one find and well written ticket
4. asfimport commented on Jan 30, 2021

   

   asfimport

   on Jan 30, 2021

   Author

   More actions

   Jorge Leitão / @jorgecarleitao:
   Issue resolved by pull request 9211
   #9211
5. asfimport

   closed this as completedon Jan 30, 2021
6. asfimport

   assigned

   jorgecarleitao

   on Jan 10, 2023
7. asfimport

   added this to the 4.0.0 milestone on Jan 11, 2023