Difference between revisions of "Installing and using KDB / Kx"
Jump to navigation
Jump to search
| (3 intermediate revisions by the same user not shown) | |||
| Line 53: | Line 53: | ||
q) | q) | ||
</pre> | </pre> | ||
| + | |||
| + | To exit the q) shell use \\ | ||
| + | <pre> | ||
| + | q)\\ | ||
| + | [root@deploy-ext q]# | ||
| + | </pre> | ||
| + | |||
| + | == Working with files / IO == | ||
| + | Read a file | ||
| + | <pre> | ||
| + | [root@deploy-ext q]# cat /root/q/data/solong.txt | ||
| + | So long | ||
| + | and thanks | ||
| + | for all the fish | ||
| + | [root@deploy-ext q]# numactl --interleave=all --physcpubind=0,1,2 l64/q | ||
| + | KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems | ||
| + | l64/ 3(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 EXPIRE 2022.05.17 david@petagene.com KOD #4176253 | ||
| + | |||
| + | q)read0 `:/root/q/data/solong.txt | ||
| + | "So long" | ||
| + | "and thanks" | ||
| + | "for all the fish" | ||
| + | </pre> | ||
| + | |||
| + | Lets try with CUNO | ||
| + | <pre> | ||
| + | [root@deploy-ext q]# numactl --interleave=all --physcpubind=0,1,2 l64/q | ||
| + | KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems | ||
| + | l64/ 3(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 EXPIRE 2022.05.17 david@petagene.com KOD #4176253 | ||
| + | |||
| + | q)read0 `:s3://bucket2/kdb-test/solong.txt | ||
| + | '/ | ||
| + | [0] read0 `:s3://bucket2/kdb-test/solong.txt | ||
| + | ^ | ||
| + | q)read0 `:s3\://bucket2/kdb-test/solong.txt | ||
| + | '/ | ||
| + | [0] read0 `:s3\://bucket2/kdb-test/solong.txt | ||
| + | ^ | ||
| + | q)read0 `:/pgcloud/s3/bucket2/kdb-test/solong.txt | ||
| + | '/ | ||
| + | [0] read0 `:/pgcloud/s3/bucket2/kdb-test/solong.txt | ||
| + | ^ | ||
| + | q)\\ | ||
| + | |||
| + | [root@deploy-ext q]# export PetaLinkMode='uricompat=q' | ||
| + | [root@deploy-ext q]# numactl --interleave=all --physcpubind=0,1,2 l64/q | ||
| + | KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems | ||
| + | l64/ 3(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 EXPIRE 2022.05.17 david@petagene.com KOD #4176253 | ||
| + | |||
| + | q)h:hopen `:/pgcloud/s3/bucket2/kdb-test/solong.txt | ||
| + | '/ | ||
| + | [0] h:hopen `:/pgcloud/s3/bucket2/kdb-test/solong.txt | ||
| + | </pre> | ||
| + | |||
| + | References: | ||
| + | * https://code.kx.com/q4m3/11_IO/#1142-using-hopen-and-hclose | ||
| + | * https://code.kx.com/q4m3/11_IO/#114-text-data | ||
| + | * https://code.kx.com/q/basics/files/ | ||
Latest revision as of 11:50, 21 May 2021
Installation
[root@deploy-ext kdb]# unzip l64.zip Archive: l64.zip inflating: l64/q inflating: q.k
Copy over the license
[root@deploy-ext q]# pwd /root/q [root@deploy-ext q]# tree . ├── kc.lic ├── l64 │ └── q ├── l64.zip └── q.k 1 directory, 4 files
License was only for 16 cores - so we need to use numa to run
[root@deploy-ext kdb]# l64/q KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems l64/ 40(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 david@petagene.com KOD #4176253 'licence error: cores
Using affinity to run Kx
# Check NUMA support [root@deploy-ext q]# grep NUMA=y /boot/config-`uname -r` CONFIG_NUMA=y # If NUMA is not enabled [root@deploy-ext q]# taskset -c 0,1 l64/q KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems l64/ 2(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 EXPIRE 2022.05.17 david@petagene.com KOD #4176253 q) # If NUMA is enabled [root@deploy-ext q]# echo 0 > /proc/sys/vm/zone_reclaim_mode [root@deploy-ext q]# numactl --interleave=all --physcpubind=0,1,2 l64/q KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems l64/ 3(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 EXPIRE 2022.05.17 david@petagene.com KOD #4176253 q)
To exit the q) shell use \\
q)\\ [root@deploy-ext q]#
Working with files / IO
Read a file
[root@deploy-ext q]# cat /root/q/data/solong.txt So long and thanks for all the fish [root@deploy-ext q]# numactl --interleave=all --physcpubind=0,1,2 l64/q KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems l64/ 3(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 EXPIRE 2022.05.17 david@petagene.com KOD #4176253 q)read0 `:/root/q/data/solong.txt "So long" "and thanks" "for all the fish"
Lets try with CUNO
[root@deploy-ext q]# numactl --interleave=all --physcpubind=0,1,2 l64/q
KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems
l64/ 3(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 EXPIRE 2022.05.17 david@petagene.com KOD #4176253
q)read0 `:s3://bucket2/kdb-test/solong.txt
'/
[0] read0 `:s3://bucket2/kdb-test/solong.txt
^
q)read0 `:s3\://bucket2/kdb-test/solong.txt
'/
[0] read0 `:s3\://bucket2/kdb-test/solong.txt
^
q)read0 `:/pgcloud/s3/bucket2/kdb-test/solong.txt
'/
[0] read0 `:/pgcloud/s3/bucket2/kdb-test/solong.txt
^
q)\\
[root@deploy-ext q]# export PetaLinkMode='uricompat=q'
[root@deploy-ext q]# numactl --interleave=all --physcpubind=0,1,2 l64/q
KDB+ 4.0 2021.04.26 Copyright (C) 1993-2021 Kx Systems
l64/ 3(16)core 128545MB root deploy-ext.dt.internal 192.168.10.1 EXPIRE 2022.05.17 david@petagene.com KOD #4176253
q)h:hopen `:/pgcloud/s3/bucket2/kdb-test/solong.txt
'/
[0] h:hopen `:/pgcloud/s3/bucket2/kdb-test/solong.txt
References: