Hello and thanks for vertica-forums,
On a 6 node cluster running on RHEL6.4 (on unsupported LVM), one node crashed, from the error log:
Code: Select all
2014-02-03 08:41:30.205 EEcmdq:0xabf7a40 [Main] <PANIC> Received fatal signal SIGSEGV.
2014-02-03 08:41:30.205 EEcmdq:0xabf7a40 [Main] <PANIC> Info: si_code: 2, si_pid: 1793868518, si_uid: 32760, si_addr: 0x7ff86aec42e6
Here is the backtrace
Code: Select all
BEGIN BACKTRACE
Vertica Backtrace at Mon Feb 3 08:40:24 2014
-------------------------
Vertica Analytic Database v7.0.0-1 $BrandId$
vertica(v7.0.0-1) built by release@build2.verticacorp.com from releases/VER_7_0_RELEASE_BUILD_0_1_20131219@127765 on 'Thu Dec 19 8:48:04 America/New_York 2013' $BuildId$
00400000-04944000 r-xp 00000000 fd:00 150169 /opt/vertica/bin/vertica
04b44000-04dce000 rw-p 04544000 fd:00 150169 /opt/vertica/bin/vertica
04dce000-04edc000 rw-p 00000000 00:00 0
05b24000-05f55000 rw-p 00000000 00:00 0 [heap]
3379000000-3379020000 r-xp 00000000 fd:00 219 /lib64/ld-2.12.so
337921f000-3379220000 r--p 0001f000 fd:00 219 /lib64/ld-2.12.so
3379220000-3379221000 rw-p 00020000 fd:00 219 /lib64/ld-2.12.so
3379221000-3379222000 rw-p 00000000 00:00 0
3379400000-337958a000 r-xp 00000000 fd:00 220 /lib64/libc-2.12.so
337958a000-3379789000 ---p 0018a000 fd:00 220 /lib64/libc-2.12.so
3379789000-337978d000 r--p 00189000 fd:00 220 /lib64/libc-2.12.so
337978d000-337978e000 rw-p 0018d000 fd:00 220 /lib64/libc-2.12.so
337978e000-3379793000 rw-p 00000000 00:00 0
3379800000-3379817000 r-xp 00000000 fd:00 221 /lib64/libpthread-2.12.so
3379817000-3379a17000 ---p 00017000 fd:00 221 /lib64/libpthread-2.12.so
3379a17000-3379a18000 r--p 00017000 fd:00 221 /lib64/libpthread-2.12.so
3379a18000-3379a19000 rw-p 00018000 fd:00 221 /lib64/libpthread-2.12.so
3379a19000-3379a1d000 rw-p 00000000 00:00 0
3379c00000-3379c02000 r-xp 00000000 fd:00 224 /lib64/libdl-2.12.so
3379c02000-3379e02000 ---p 00002000 fd:00 224 /lib64/libdl-2.12.so
3379e02000-3379e03000 r--p 00002000 fd:00 224 /lib64/libdl-2.12.so
3379e03000-3379e04000 rw-p 00003000 fd:00 224 /lib64/libdl-2.12.so
337a800000-337a807000 r-xp 00000000 fd:00 233 /lib64/librt-2.12.so
337a807000-337aa06000 ---p 00007000 fd:00 233 /lib64/librt-2.12.so
337aa06000-337aa07000 r--p 00006000 fd:00 233 /lib64/librt-2.12.so
337aa07000-337aa08000 rw-p 00007000 fd:00 233 /lib64/librt-2.12.so
337b800000-337b816000 r-xp 00000000 fd:00 242 /lib64/libgcc_s-4.4.7-20120601.so.1
337b816000-337ba15000 ---p 00016000 fd:00 242 /lib64/libgcc_s-4.4.7-20120601.so.1
337ba15000-337ba16000 rw-p 00015000 fd:00 242 /lib64/libgcc_s-4.4.7-20120601.so.1
337c800000-337c822000 r-xp 00000000 fd:00 292 /lib64/libncurses.so.5.7
337c822000-337ca21000 ---p 00022000 fd:00 292 /lib64/libncurses.so.5.7
337ca21000-337ca22000 rw-p 00021000 fd:00 292 /lib64/libncurses.so.5.7
337cc00000-337cc1d000 r-xp 00000000 fd:00 259 /lib64/libtinfo.so.5.7
337cc1d000-337ce1d000 ---p 0001d000 fd:00 259 /lib64/libtinfo.so.5.7
337ce1d000-337ce21000 rw-p 0001d000 fd:00 259 /lib64/libtinfo.so.5.7
7ffe4c2fd000-7ffe5218e000 r--p 00000000 fd:00 139072 /usr/lib/locale/locale-archive
7ffe5218e000-7ffe530d3000 r--s 00000000 fd:00 273339 /opt/vertica/share/icu/icudt42l.dat
7ffe530d3000-7ffe53108000 r--s 00000000 fd:02 299 /var/db/nscd/group
7ffe53108000-7ffe63268000 rw-p 00000000 00:00 0
7ffe63268000-7ffe63468000 rw-p 00000000 00:00 0
7ffe63468000-7ffe6349d000 r--s 00000000 fd:02 300 /var/db/nscd/hosts
7ffe6349d000-7ffe636b1000 rw-p 00000000 00:00 0
7ffe636b1000-7ffe636e6000 r--s 00000000 fd:02 293 /var/db/nscd/passwd
7ffe636e6000-7ffe636eb000 rw-p 00000000 00:00 0
7ffe636ee000-7ffe636fb000 rw-p 00000000 00:00 0
7fffef5fe000-7fffef615000 rw-p 00000000 00:00 0 [stack]
7fffef6a0000-7fffef6a1000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Backtrace Generated by Error
Signal: [0x000000000000000b] PID: [0x0000000000002f6a] PC: [0x000000000167e624] FP: [0x00007ffb67d406f0] SIGSEGV: SEGV_ACCERR SI_ADDR : [0x00007ff86aec42e6]
/opt/vertica/bin/vertica(_ZN6Basics9Backtrace11DoBacktraceEiiPvS1_+0x8cc)[0x3370fee]
/opt/vertica/bin/vertica(_ZN6Basics20GlobalSignalHandlers14logFatalSignalEiPvS1_+0xc7)[0x33f0205]
/opt/vertica/bin/vertica[0x33f07f3]
/lib64/libc.so.6[0x3379432920]
/opt/vertica/bin/vertica(_ZN2EE11GroupByHash3runEv+0xf9e)[0x167e624]
/opt/vertica/bin/vertica[0x15d4808]
/lib64/libc.so.6[0x3379443b70]
END BACKTRACE
THREAD CONTEXT
Thread type: EE Internal Command Queue Thread
Request: INSERT INTO schema_a.table_a (SELECT DISTINCT fields...,1 FROM schema_a.table_a);DELETE FROM schema_a.table_a WHERE keep_col IS NULL;ALTER TABLE schema_a.table_a DROP keep_col;
134: Send
6: FifoBuffer
19: Router
18: ValExpr
143: Copy
FAULT => 33: NewEENode
34) ExprEval depth=(0) parent=0, peer#0; outTup nCol=16,nkey=0,inlSz=281,fixSz=128
35) ExprEval (1) 34, #0; out 15,0,273,120
36) ParallelMerge (2) 35, #0; nInputs=16; out 14,0,265,112
37) GroupByHash (3) 36, #0; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
38) StorageUnion (4) 37, #0; nInputs=28; out 14,0,265,112
39) GroupByPipe (5) 38, #0; out 14,0,265,112
40) Scan (6) 39, #0; out 14,0,265,112
41) GroupByPipe (5) 38, #1; out 14,0,265,112
42) Scan (6) 41, #1; out 14,0,265,112
43) GroupByPipe (5) 38, #2; out 14,0,265,112
44) Scan (6) 43, #2; out 14,0,265,112
45) GroupByPipe (5) 38, #3; out 14,0,265,112
46) Scan (6) 45, #3; out 14,0,265,112
47) GroupByPipe (5) 38, #4; out 14,0,265,112
48) Scan (6) 47, #4; out 14,0,265,112
49) GroupByPipe (5) 38, #5; out 14,0,265,112
50) Scan (6) 49, #5; out 14,0,265,112
51) GroupByPipe (5) 38, #6; out 14,0,265,112
52) Scan (6) 51, #6; out 14,0,265,112
53) GroupByPipe (5) 38, #7; out 14,0,265,112
54) Scan (6) 53, #7; out 14,0,265,112
55) GroupByPipe (5) 38, #8; out 14,0,265,112
56) Scan (6) 55, #8; out 14,0,265,112
57) GroupByPipe (5) 38, #9; out 14,0,265,112
58) Scan (6) 57, #9; out 14,0,265,112
59) GroupByPipe (5) 38, #10; out 14,0,265,112
60) Scan (6) 59, #10; out 14,0,265,112
61) GroupByPipe (5) 38, #11; out 14,0,265,112
62) Scan (6) 61, #11; out 14,0,265,112
63) GroupByPipe (5) 38, #12; out 14,0,265,112
64) Scan (6) 63, #12; out 14,0,265,112
65) GroupByPipe (5) 38, #13; out 14,0,265,112
66) Scan (6) 65, #13; out 14,0,265,112
67) GroupByPipe (5) 38, #14; out 14,0,265,112
68) Scan (6) 67, #14; out 14,0,265,112
69) GroupByPipe (5) 38, #15; out 14,0,265,112
70) Scan (6) 69, #15; out 14,0,265,112
71) GroupByPipe (5) 38, #16; out 14,0,265,112
72) Scan (6) 71, #16; out 14,0,265,112
73) GroupByPipe (5) 38, #17; out 14,0,265,112
74) Scan (6) 73, #17; out 14,0,265,112
75) GroupByPipe (5) 38, #18; out 14,0,265,112
76) Scan (6) 75, #18; out 14,0,265,112
77) GroupByPipe (5) 38, #19; out 14,0,265,112
78) Scan (6) 77, #19; out 14,0,265,112
79) GroupByPipe (5) 38, #20; out 14,0,265,112
80) Scan (6) 79, #20; out 14,0,265,112
81) GroupByPipe (5) 38, #21; out 14,0,265,112
82) Scan (6) 81, #21; out 14,0,265,112
83) GroupByPipe (5) 38, #22; out 14,0,265,112
84) Scan (6) 83, #22; out 14,0,265,112
85) GroupByPipe (5) 38, #23; out 14,0,265,112
86) Scan (6) 85, #23; out 14,0,265,112
87) GroupByPipe (5) 38, #24; out 14,0,265,112
88) Scan (6) 87, #24; out 14,0,265,112
89) GroupByPipe (5) 38, #25; out 14,0,265,112
90) Scan (6) 89, #25; out 14,0,265,112
91) GroupByPipe (5) 38, #26; out 14,0,265,112
92) Scan (6) 91, #26; out 14,0,265,112
93) GroupByPipe (5) 38, #27; out 14,0,265,112
94) Scan (6) 93, #27; out 14,0,265,112
95) GroupByHash (3) 36, #1; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
96) StorageUnion (4) 95, #1; out 14,0,265,112
97) GroupByHash (3) 36, #2; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
98) StorageUnion (4) 97, #2; out 14,0,265,112
99) GroupByHash (3) 36, #3; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
100) StorageUnion (4) 99, #3; out 14,0,265,112
101) GroupByHash (3) 36, #4; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
102) StorageUnion (4) 101, #4; out 14,0,265,112
103) GroupByHash (3) 36, #5; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
104) StorageUnion (4) 103, #5; out 14,0,265,112
105) GroupByHash (3) 36, #6; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
106) StorageUnion (4) 105, #6; out 14,0,265,112
107) GroupByHash (3) 36, #7; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
108) StorageUnion (4) 107, #7; out 14,0,265,112
(PPFAULT) => GroupByHash (id=109) (3) 36, #8; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
110) StorageUnion (4) 109, #8; out 14,0,265,112
111) GroupByHash (3) 36, #9; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
112) StorageUnion (4) 111, #9; out 14,0,265,112
113) GroupByHash (3) 36, #10; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
114) StorageUnion (4) 113, #10; out 14,0,265,112
115) GroupByHash (3) 36, #11; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
116) StorageUnion (4) 115, #11; out 14,0,265,112
117) GroupByHash (3) 36, #12; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
118) StorageUnion (4) 117, #12; out 14,0,265,112
119) GroupByHash (3) 36, #13; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
120) StorageUnion (4) 119, #13; out 14,0,265,112
121) GroupByHash (3) 36, #14; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
122) StorageUnion (4) 121, #14; out 14,0,265,112
123) GroupByHash (3) 36, #15; CRPstate=2;hash sca=18; kvlen=(112,0) out 14,0,265,112
124) StorageUnion (4) 123, #15; out 14,0,265,112
Transaction: [0x00f000000e82e615]
END THREAD CONTEXT
I obfuscated the queries.
We are enterprise customers but I unable to login to https://my.vertica.com using my work email address, password resets do not seem to help, I have requested a new account using a different email address. My posts to https://community.vertica.com/vertica are also failing at the point of preview.
Thanks,
Peter (Stig) Edwards