Returning rows from R UDx is really slow

Moderator: NorbertKrupa

rytaft
Newbie
Newbie
Posts: 13
Joined: Thu Jan 17, 2013 5:14 pm

Returning rows from R UDx is really slow

Post by rytaft » Mon Mar 25, 2013 5:44 am

I am writing a simple user defined transform function in R to compute matrix multiplication. Here is the code:

Code: Select all

mmult_udf <- function(x)
{
        ptm <- proc.time()
        library(reshape2)
        library(Matrix)
        sm = sparseMatrix(x[,1], x[,2], x=x[,3])
        A = as.matrix(sm)

        S = t(A)%*%A
        res <- melt(S)
        print(proc.time() - ptm)
        res
}

matrixMultiplyFactory <- function()
{
        list(name=mmult_udf,udxtype=c("transform"),intype=c("int","int","float"), outtype=c("int","int","float"), 
                outnames=c("row_num", "col_num", "val"))
}
As shown above, I print out the time it takes to complete the R code alone. When I tried running this on a 12k x 1k size dataset, the R code completed in about one minute, but the rows did not actually get returned to Vertica for almost 80 minutes. Anyone have an idea about why returning/formatting rows in Vertica is so slow?

id10t
GURU
GURU
Posts: 732
Joined: Mon Apr 16, 2012 2:44 pm

Re: Returning rows from R UDx is really slow

Post by id10t » Mon Mar 25, 2013 3:59 pm

Hi!

Can you post EXPLAIN of query? Also if you can run PROFILE and see where is "bottle neck" (winch component takes most time)
Look, R do not supports in parallel execution and it's a big challenge to implement parallelism in R with Vertica.
May be there are a some "bug" in reduce step.

PS
Without EXPLAIN and PROFILE only Vertica support can answer to you.

rytaft
Newbie
Newbie
Posts: 13
Joined: Thu Jan 17, 2013 5:14 pm

Re: Returning rows from R UDx is really slow

Post by rytaft » Mon Mar 25, 2013 11:47 pm

Here is the SQL code I am running:

Code: Select all

DROP LIBRARY RFunctions CASCADE;
CREATE LIBRARY RFunctions AS '/home/dbadmin/vertica-R-functions.R' LANGUAGE 'R';
CREATE TRANSFORM FUNCTION mmult AS LANGUAGE 'R' NAME 'matrixMultiplyFactory' LIBRARY RFunctions;

SELECT mmult(row_num, col_num, vals) OVER() FROM exp_data;
Here is what I get when I run EXPLAIN on the query:


QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------
QUERY PLAN DESCRIPTION:
------------------------------

EXPLAIN SELECT mmult(row_num, col_num, vals) OVER() FROM exp_data;

Access Path:
+-ANALYTICAL [Cost: 146K, Rows: 12M] (PATH ID: 1)
| Analytic Group
| Functions: mmult()
| +---> STORAGE ACCESS for exp_data [Cost: 84K, Rows: 12M] (PATH ID: 2)
| | Projection: public.exp_data_super
| | Materialize: exp_data.row_num, exp_data.col_num, exp_data.vals


------------------------------
-----------------------------------------------
PLAN: BASE QUERY PLAN (GraphViz Format)
-----------------------------------------------
digraph G {
graph [rankdir=BT, label = "BASE QUERY PLAN\nQuery: EXPLAIN SELECT mmult(row_num, col_num, vals) OVER() FROM exp_data;\n\nAll Nodes Vector: \n\n node[0]=v_dbadmin_node0001 (initiator) Up\n", labelloc=t, labeljust=l ordering=out]
0[label = "Root \nOutBlk=[UncTuple(3)]", color = "green", shape = "house"];
1[label = "NewEENode \nOutBlk=[UncTuple(3)]", color = "green", shape = "box"];
2[label = "UserDefined:\nPartitionBy:<NONE>\nOrderBy:<NONE>\nAnalytical Functions:\n mmult(exp_data.row_num, exp_data.col_num, exp_data.vals)\nUnc: Integer(8)\nUnc: Integer(8)\nUnc: Float(8)", color = "green", shape = "box"];
3[label = "StorageUnionStep: exp_data_super\nUnc: Integer(8)\nUnc: Integer(8)\nUnc: Float(8)", color = "purple", shape = "box"];
4[label = "ScanStep: exp_data_super\nrow_num\ncol_num\nvals\nUnc: Integer(8)\nUnc: Integer(8)\nUnc: Float(8)", color = "brown", shape = "box"];
1->0 [label = "V[0] C=3",color = "black",style="bold", arrowtail="inv"];
2->1 [label = "0",color = "blue"];
3->2 [label = "0",color = "blue"];
4->3 [label = "0",color = "blue"];
}
(31 rows)


And here is what I get when I run PROFILE and select the appropriate rows from the execution_engine_profiles table:

Code: Select all

     node_name      |      user_id      | user_name |            session_id             |  transaction_id   | statement_id | operator_name | operator_id | baseplan_id | path_id | localplan_id |       counter_name       | counter_tag | counter_value | is_executing 
--------------------+-------------------+-----------+-----------------------------------+-------------------+--------------+---------------+-------------+-------------+---------+--------------+--------------------------+-------------+---------------+--------------
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Root          |           0 |           0 |      -1 |            0 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Root          |           0 |           0 |      -1 |            0 | memory allocated (bytes) |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Root          |           0 |           0 |      -1 |            0 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Root          |           0 |           0 |      -1 |            0 | rows produced            |             |        885481 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Root          |           0 |           0 |      -1 |            0 | input queue wait (us)    |             |    5242936985 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Root          |           0 |           0 |      -1 |            0 | clock time (us)          |             |       2459250 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Root          |           0 |           0 |      -1 |            0 | execution time (us)      |             |       2413602 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | NewEENode     |           1 |           1 |      -1 |            1 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | NewEENode     |           1 |           1 |      -1 |            1 | memory allocated (bytes) |             |        524560 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | NewEENode     |           1 |           1 |      -1 |            1 | memory reserved (bytes)  |             |    -112295936 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | NewEENode     |           1 |           1 |      -1 |            1 | rle rows produced        |             |        885481 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | NewEENode     |           1 |           1 |      -1 |            1 | rows produced            |             |        885481 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | NewEENode     |           1 |           1 |      -1 |            1 | output queue wait (us)   |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | NewEENode     |           1 |           1 |      -1 |            1 | clock time (us)          |             |          5767 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | NewEENode     |           1 |           1 |      -1 |            1 | execution time (us)      |             |          5315 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Analytical    |           2 |           2 |       1 |            2 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Analytical    |           2 |           2 |       1 |            2 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Analytical    |           2 |           2 |       1 |            2 | memory allocated (bytes) |             |       1049792 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Analytical    |           2 |           2 |       1 |            2 | memory reserved (bytes)  |             |     109101056 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Analytical    |           2 |           2 |       1 |            2 | rle rows produced        |             |        885481 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Analytical    |           2 |           2 |       1 |            2 | rows produced            |             |        885481 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Analytical    |           2 |           2 |       1 |            2 | clock time (us)          |             |    5245292542 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Analytical    |           2 |           2 |       1 |            2 | execution time (us)      |             |    5235045774 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | request wait (us)        |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | response wait (us)       |             |          4049 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | memory allocated (bytes) |             |        705504 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | memory reserved (bytes)  |             |       3588096 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | rle rows produced        |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | rows produced            |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | clock time (us)          |             |         64539 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | StorageUnion  |           3 |           3 |       2 |            3 | execution time (us)      |             |         54717 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | bytes read from disk     |             |       1868217 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | bytes read from cache    |             |       5706144 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | rows processed           |             |        346730 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | memory allocated (bytes) |             |        919128 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | rle rows produced        |             |        346730 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | rows produced            |             |        346730 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | clock time (us)          |             |         39912 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          15 |           4 |       2 |            4 | execution time (us)      |             |         37434 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | bytes read from disk     |             |       5697208 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | bytes read from cache    |             |       3473408 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | memory allocated (bytes) |             |       1082968 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | clock time (us)          |             |        105988 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          14 |           4 |       2 |            4 | execution time (us)      |             |        102360 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | bytes read from disk     |             |       6796640 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | bytes read from cache    |             |       1703936 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | memory allocated (bytes) |             |       1082968 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | clock time (us)          |             |        121878 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          13 |           4 |       2 |            4 | execution time (us)      |             |        119521 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | bytes read from disk     |             |       6356157 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | bytes read from cache    |             |       2359296 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | memory allocated (bytes) |             |       1082968 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | clock time (us)          |             |        116006 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          12 |           4 |       2 |            4 | execution time (us)      |             |        112868 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | bytes read from disk     |             |       4052471 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | bytes read from cache    |             |      19267584 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | memory allocated (bytes) |             |       1017432 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | clock time (us)          |             |         92463 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          11 |           4 |       2 |            4 | execution time (us)      |             |         88464 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | bytes read from disk     |             |       6736766 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | bytes read from cache    |             |       1966080 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | memory allocated (bytes) |             |       1082968 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | clock time (us)          |             |        124526 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |          10 |           4 |       2 |            4 | execution time (us)      |             |        119726 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | bytes read from disk     |             |       5959440 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | bytes read from cache    |             |       2949120 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | memory allocated (bytes) |             |       1082968 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | clock time (us)          |             |        112291 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           9 |           4 |       2 |            4 | execution time (us)      |             |        107562 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | bytes read from disk     |             |       6549553 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | bytes read from cache    |             |       2031616 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | memory allocated (bytes) |             |       1082968 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | clock time (us)          |             |        120650 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           8 |           4 |       2 |            4 | execution time (us)      |             |        116774 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | bytes read from disk     |             |       4707246 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | bytes read from cache    |             |      18350080 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | memory allocated (bytes) |             |       1017432 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | clock time (us)          |             |        100613 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           7 |           4 |       2 |            4 | execution time (us)      |             |         96575 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | bytes read from disk     |             |       5744301 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | bytes read from cache    |             |       3342336 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | memory allocated (bytes) |             |       1082968 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | clock time (us)          |             |        112658 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           6 |           4 |       2 |            4 | execution time (us)      |             |        107773 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | bytes read from disk     |             |       6731657 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | bytes read from cache    |             |       1769472 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | memory allocated (bytes) |             |       1082968 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | clock time (us)          |             |        124751 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           5 |           4 |       2 |            4 | execution time (us)      |             |        120267 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | bytes read from disk     |             |       6784965 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | bytes read from cache    |             |       1572864 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | rows pruned by valindex  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | rows processed           |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | estimated rows produced  |             |      11881066 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | file handles             |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | memory allocated (bytes) |             |       1074776 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | memory reserved (bytes)  |             |             0 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | rle rows produced        |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | rows produced            |             |       1048576 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | clock time (us)          |             |        125606 | f
 v_dbadmin_node0001 | 45035996273704962 | dbadmin   | vise3.csail.mit.edu-17090:0x28fb4 | 45035996273788846 |            2 | Scan          |           4 |           4 |       2 |            4 | execution time (us)      |             |        121846 | f
(176 rows)


Any help would be much appreciated!

id10t
GURU
GURU
Posts: 732
Joined: Mon Apr 16, 2012 2:44 pm

Re: Returning rows from R UDx is really slow

Post by id10t » Tue Mar 26, 2013 10:07 am

Hi!

Ok. What we got:

What takes so long ?

Code: Select all

daniel=> select operator_name,counter_name,counter_value/(6*10^7) as 'MINUTES' from qp  where counter_name ilike '%clock%' order by 3 desc;
 operator_name | counter_name  |       MINUTES        
---------------+---------------+----------------------
 Analytical    | clocktime(us) |     87.4215423666667
 Root          | clocktime(us) |            0.0409875
 Scan          | clocktime(us) |  0.00209343333333333
 Scan          | clocktime(us) |  0.00207918333333333
 Scan          | clocktime(us) |  0.00207543333333333
 Scan          | clocktime(us) |            0.0020313
 Scan          | clocktime(us) |  0.00201083333333333
 Scan          | clocktime(us) |  0.00193343333333333
 Scan          | clocktime(us) |  0.00187763333333333
 Scan          | clocktime(us) |  0.00187151666666667
 Scan          | clocktime(us) |  0.00176646666666667
 Scan          | clocktime(us) |  0.00167688333333333
 Scan          | clocktime(us) |           0.00154105
 StorageUnion  | clocktime(us) |           0.00107565
 StorageUnion  | clocktime(us) |           0.00107565
 Scan          | clocktime(us) |            0.0006652
 NewEENode     | clocktime(us) | 9.61166666666667e-05
Hm... Analytical: ~90 minutes. Nice.

Next:

Code: Select all

daniel=> select operator_name, counter_name, counter_value from qp where operator_name = 'Analytical';
 operator_name |      counter_name      | counter_value 
---------------+------------------------+---------------
 Analytical    | clocktime(us)          |    5245292542
 Analytical    | estimatedrowsproduced  |      11881066
 Analytical    | executiontime(us)      |    5235045774
 Analytical    | filehandles            |             0
 Analytical    | memoryallocated(bytes) |       1049792
 Analytical    | memoryreserved(bytes)  |     109101056
 Analytical    | rlerowsproduced        |        885481
 Analytical    | rowsproduced           |        885481
RLE - ok, MEMORY - ok, ROWS - OK:12Mil (small data set), no spills to disk.

Only one issue I have to you: Projection: public.exp_data_super.
Run DBD on table + query and post a results after it.


PS
There are only one "Analytical" operator. Do you have only one node? If so try R + Presto!
Hm.. May be sparse matrix creates a new table in Vertica?
"Analytical" - says nothing, since you don't know how Vertica works with R in low level.

rytaft
Newbie
Newbie
Posts: 13
Joined: Thu Jan 17, 2013 5:14 pm

Re: Returning rows from R UDx is really slow

Post by rytaft » Tue Mar 26, 2013 3:40 pm

Thanks so much for taking the time to investigate this! Here is output from running Database Designer:


Database Designer started.

For large databases a design session could take a long time; allow it to complete uninterrupted.
Use Ctrl+C if you must cancel the session.

Setting up design session...

Examining table data...

Loading queries from '/home/dbadmin/vertica-basic-test.sql'.
Processed 6 SQL statement(s), only 2 accepted and considered in the design.
Identified issues:
4 SQL statement(s) are not SELECT queries - REJECTED.

Query processing problems are logged in designer.log. The first problem is...

4994: This non-SELECT query is not supported: CREATE LIBRARY RFunctions AS '/home/dbadmin/vertica-R-basic-functions-tuned.R' LANGUAGE 'R';


Saving existing projections to /home/dbadmin/dbadmin_design_projection_backup_0326140929.sql ...
[100%] Saving existing projection definitions... 3 of 3


Creating design and deploying projections...
[100%] Optimizing storage footprint... Completed of .

Query optimization results...ress...

Query 1 optimization ratio or status is CANNOT BE OPTIMIZED DUE TO QUERY TYPE
Query 2 optimization ratio or status is CANNOT BE OPTIMIZED DUE TO QUERY TYPE


Deploying and generating deployment script...
[100%] Deploying/Dropping projections... Completed 6 of 6 projections.

Completed 6 of 6 projections.
Design script is located in /home/dbadmin/dbadmin_design_design.sql
Deployment script is located in /home/dbadmin/dbadmin_design_deploy.sql

Database Designer finished.

----------------

Based on the above, it looks like the Database Designer cannot optimize queries that depend on R UDx code. I am re-running the query now and it doesn't seem to be any faster.
----------------

Regarding your question - yes I am only running on one node, but this seems unusually slow even for just one node. By printing out the timestamp I have determined that the real "analytical" work completes in less than one minute, and it's just the formatting/returning of the rows that is somehow taking a long time. I have tried both sending the output to a file and sending output to /dev/null. When I send the output to a file, I start seeing results after only one minute, but it takes 80-90 minutes for all the results to be printed. Sending output to /dev/null doesn't seem to save any time -- Vertica seems to be doing the same amount of work. Any ideas you have are appreciated!

Thanks!

id10t
GURU
GURU
Posts: 732
Joined: Mon Apr 16, 2012 2:44 pm

Re: Returning rows from R UDx is really slow

Post by id10t » Tue Mar 26, 2013 7:44 pm

Hi!

1. With one node + matrix analytics, like matrix "transpose", don't use in Vertica, use in pure R.
2. For fetching data from Vertica use in RODBC or RJDBC drivers.

Vertica achievement with R its a parallel calculation/analytics, since R has no MPP (yet).
So using Vertica with R on one node (especially for matrix analytics) has no advantages, only disadvantages. Use in Vertica only as fast storage with calculation on rows only.
Also (forget about a problem of matrix transpose): without sorting a data Vertica work bad and you can not sort all matrices, sometimes order or rows is important :-) Be careful with Vertica - for all tables you have hold a column with row number, to avoid reordering.

PS
>> but this seems unusually slow even for just one node.
You are right and if you can open a ticket.
But... matrices analytics on one node + R..., I dont believe you will get results faster than in {pure R} + {COPY a result to Vertica}.

IMHO: InfoBright will be better with matrix analytics on one node - this db based "rough set".

rytaft
Newbie
Newbie
Posts: 13
Joined: Thu Jan 17, 2013 5:14 pm

Re: Returning rows from R UDx is really slow

Post by rytaft » Tue Mar 26, 2013 8:00 pm

Thanks for all this info! Any idea how I can open a ticket? I am using the Vertica Community Edition. It doesn't seem like they pay attention to their forum since I posted something 2 weeks ago and never got a response.

Post Reply

Return to “R Language Integration”