Page 1 of 1
Partitions and ROS Containers
Posted: Fri Nov 13, 2015 4:47 pm
by beth
Hi all,
From the admin guide I read this:
Partitions and ROS Containers
1. Data is automatically split into partitions during load / refresh / recovery operations.
2. The Tuple Mover maintains physical separation of partitions.
3. Each ROS container contains data for a single partition, though there can be multiple ROS containers for a single partition.
So for #3, under what circumstances would there be "multiple ROS containers for a single partition"?
Re: Partitions and ROS Containers
Posted: Fri Nov 13, 2015 5:08 pm
by JimKnicely
Prior to a MERGEOUT operation, you could see extra ROS containers for partitioned data.
Simple example:
Code: Select all
dbadmin=> drop table test;
DROP TABLE
dbadmin=> create table test (c1 int not null, c2 int) partition by c1;
CREATE TABLE
dbadmin=> insert /*+ direct */ into test values (1, 1);
OUTPUT
--------
1
(1 row)
dbadmin=> insert /*+ direct */ into test values (2, 1);
OUTPUT
--------
1
(1 row)
After the inserts, we have two ROS containers (see ros_count column):
Code: Select all
dbadmin=> select projection_name, ros_row_count, ros_count from projection_storage where anchor_table_name = 'test';
projection_name | ros_row_count | ros_count
-----------------+---------------+-----------
test_super | 2 | 2
(1 row)
Code: Select all
dbadmin=> insert /*+ direct */ into test values (1, 1);
OUTPUT
--------
1
(1 row)
dbadmin=> insert /*+ direct */ into test values (2, 1);
OUTPUT
--------
1
(1 row)
After these inserts, we now have four ROS containers. Vertica did not at this point combine the partition data into a single container for each partition:
Code: Select all
dbadmin=> select projection_name, ros_row_count, ros_count from projection_storage where anchor_table_name = 'test';
projection_name | ros_row_count | ros_count
-----------------+---------------+-----------
test_super | 4 | 4
(1 row)
dbadmin=> select do_tm_task('mergeout', 'test');
do_tm_task
----------------------------------------------------------------------
Task: mergeout
(Table: public.test) (Projection: public.test_super)
(1 row)
But after a MERGEOUT, Vertica did combine (i.e. merge) the partitioned data into a single container for each partition:
Code: Select all
dbadmin=> select projection_name, ros_row_count, ros_count from projection_storage where anchor_table_name = 'test';
projection_name | ros_row_count | ros_count
-----------------+---------------+-----------
test_super | 4 | 2
(1 row)
Make sense?
Re: Partitions and ROS Containers
Posted: Fri Nov 13, 2015 8:08 pm
by scutter
A few additional comments beyond what Jim wrote.
- The “active” partitions (1 or more) will have more than one ROS container, until they become “past partitions”
- Past partitions may have more than one ROS container. Originally the TM would aggressively merge past partitions into a single ROS container, but in current versions it won’t always merge multiple ROS containers in past partitions to a single ROS. If you have a ROS container in a past partition it’s not necessary worthwhile to merge it with a large ROS, so the TM won’t do that.
- DVROSes (for delete vectors) may also co-exist and count towards the number of ROS containers.
- I’ve observed that recent versions have a higher limit for ROS containers than the technical limit. For example if the limit in the configuration parameter is 1024, it lets you go to 2048. I haven’t spotted anything in the documentation for this, so there’s no commitment that this would remain true, but it does give you more head room.
—Sharon
Re: Partitions and ROS Containers
Posted: Sat Nov 14, 2015 4:34 pm
by beth
Thank you Jim and Sharon! Your posts are very, very helpful!!!