Difference between revisions of "VScaler: Advantech PCIe passthrough"

From Define Wiki
Jump to navigation Jump to search
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
−
The setup :
+
== The setup ==
  
 
* Intel(R) Xeon(R) CPU E3-1225 v5 @ 3.30GHz
 
* Intel(R) Xeon(R) CPU E3-1225 v5 @ 3.30GHz
 
* 16 GB RAM  
 
* 16 GB RAM  
 
* Advantech Co. Ltd Device M30 card and fpga.
 
* Advantech Co. Ltd Device M30 card and fpga.
 +
* Base OS: centos 7.6
 +
* OPenstack version queens
 +
* backend storage ceph
 +
 +
== vScaler setup and passthrough ==
  
 
We have setup an ALL in one scaler setup to passthrough the advantech card. The globals files for the installation is attached.  
 
We have setup an ALL in one scaler setup to passthrough the advantech card. The globals files for the installation is attached.  
Line 23: Line 28:
 
enable_haproxy: "no"
 
enable_haproxy: "no"
  
 +
</syntaxhighlight>
  
−
09:00.0 Multimedia controller: Advantech Co. Ltd Device 004b (rev 01)
+
 
 +
The advantech card has 2 parts M30 codec and fpga part . check in lscpi -s
 +
 
 +
<syntaxhighlight>
 +
[root@localhost ~]#lspci -s 09:00.0  
 +
Multimedia controller: Advantech Co. Ltd Device 004b (rev 01)
 +
 
 +
[root@localhost ~]#lspci -s 0b:00.0
 
0b:00.0 Multimedia controller: Device 50c1:0003 (rev 01)
 
0b:00.0 Multimedia controller: Device 50c1:0003 (rev 01)
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 +
Advantech cards pass-through works with the following set of QEMU* version of rpms. Download the same rpm to make the passthrough work
  
−
Advatech cards passthrough works with the following set of QEMU* version of rpms.  
+
Note : the other Qemu* rpm will end the system in Hang mode and restart after creating virtual machine.
  
 
<syntaxhighlight>
 
<syntaxhighlight>
Line 59: Line 73:
  
 
</syntaxhighlight>
 
</syntaxhighlight>
−
The containers with Qemu* and QEMU-kvm* rpms are nova_libvirt , nova_compute , glance_api .. The changes were not needed in qemu-img-ev* in cinder* and ceph containers.
 
  
−
<syntaxhighlight>
+
The containers with Qemu* and QEMU-kvm* rpms are nova_libvirt , nova_compute , glance_api .
 +
The changes were not needed in qemu-img-ev* in cinder* and ceph containers.
 +
 
 +
== Configure nova-api (Controller) ==
 +
 
 +
Specify the PCI alias for the device.
  
 +
Change the nova.conf libvirt are mandatory. Add the pci_white list information to while list the
  
−
</syntaxhighlight>
 
  
 
<syntaxhighlight>
 
<syntaxhighlight>
Line 87: Line 105:
  
 
<syntaxhighlight>
 
<syntaxhighlight>
 +
 
#check nova-scheduler , nova-api , nova-compute all have the setting
 
#check nova-scheduler , nova-api , nova-compute all have the setting
  
Line 117: Line 136:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
docker
+
 
 +
== Reconfgigure the Kolla ansible setup to mreflect the changes
 +
 
 +
<syntaxhighlight>
 +
[root@localhost ~]# kolla-ansible -i all_in_one reconfigure
 +
</syntaxhighlight>
  
  
Line 289: Line 313:
 
Kernel modules: vega3311_fpga
 
Kernel modules: vega3311_fpga
  
−
[16:13] Shanal Tiwari
 
−
   
 
  
−
[root@mediaflow-bioslab vega3311_fpgautil]# vega3311_cap_app list_dev
+
</syntaxhighlight>
 +
 
 +
To verify the drivers are installed correctly chyeck the following command
 +
 
 +
<syntaxhighlight>
  
 +
[root@mediaflow-bioslab scratch]# vega3311_cap_app list_dev
 
DRV INF: SF: Got idcode 0xc2 0x20 0x1a
 
DRV INF: SF: Got idcode 0xc2 0x20 0x1a
−
 
 
Detected 1 chip(s).
 
Detected 1 chip(s).
−
 
 
==============================================
 
==============================================
−
 
 
Chip: 0
 
Chip: 0
−
 
+
Device node name: /dev/vega3311_fpga_0
−
Device node name: /dev/vega3311_fpga_0
+
PCI Bridge 0x08
−
 
+
PCI Bus:Slot.Func 09:00.0
−
PCI Bridge 0x08
+
Vendor ID: 0x13fe Device ID: 0x004b
−
 
+
Subsystem VendorID: 0x13fe Subsystem DevID: 0x004b
−
PCI Bus:Slot.Func 09:00.0
+
Serial Number: LKD0114896 Board ID: 9792A33111E
−
 
+
Irq: 18
−
Vendor ID: 0x13fe Device ID: 0x004b
+
BAR Configuration:
−
 
+
Start              | Length
−
Subsystem VendorID: 0x13fe Subsystem DevID: 0x004b
+
0x00000000de800000 | 000131072
−
 
+
0x00000000de400000 | 004194304
−
Serial Number: LKD0114896 Board ID: 9792A33111E
 
−
 
 
−
Irq: 18
 
−
 
 
−
BAR Configuration:
 
−
 
 
−
Start              | Length
 
−
 
 
−
0x00000000de800000 | 000131072
 
−
 
 
−
0x00000000de400000 | 004194304
 
−
 
 
 
==============================================
 
==============================================
−
 
+
[root@mediaflow-bioslab scratch]#
−
[root@mediaflow-bioslab vega3311_fpgautil]#  
+
[root@mediaflow-bioslab scratch]# vega3311_fpgautil list
−
 
+
[Card 0] Device id 0, PCIe bridge 8
−
​[16:13] Shanal Tiwari
 
 
      
 
      
−
 
−
[root@mediaflow-bioslab vega3311_fpgautil]# vega3311_fpgautil list
 
−
 
−
[Card 0] Device id 0, PCIe bridge 8
 
−
 
−
​[16:14] Shanal Tiwari
 
−
   
 
−
 
−
[root@mediaflow-bioslab vega3311_fpgautil]# vega3311_fpgautil qpcie 0
 
−
 
−
DRV INF: SF: Got idcode 0xc2 0x20 0x1a
 
−
 
−
==============================================
 
−
 
−
Chip: 0
 
−
 
−
Device node name: /dev/vega3311_fpga_0
 
−
 
−
PCI Bridge 0x08
 
−
 
−
PCI Bus:Slot.Func 09:00.0
 
−
 
−
Vendor ID: 0x13fe Device ID: 0x004b
 
−
 
−
Subsystem VendorID: 0x13fe Subsystem DevID: 0x004b
 
−
 
−
Class: 0x00048000
 
−
 
−
Header Type: 0 Irq: 18
 
−
 
−
BAR Configuration:
 
−
 
−
Start              | Length    | Flags
 
−
 
−
0x00000000de800000 | 000131072 | 0x00040200
 
−
 
−
0x00000000de400000 | 004194304 | 0x00040200
 
−
 
−
0x0000003fff800000 | 004194304 | 0x0014220c
 
−
 
−
bootup image: 1st image (3G SDI)
 
−
 
−
working image: 1st image (3G SDI)
 
−
 
−
==============================================
 
−
 
  
 
</syntaxhighlight>
 
</syntaxhighlight>

Latest revision as of 17:41, 21 February 2020

The setup

  • Intel(R) Xeon(R) CPU E3-1225 v5 @ 3.30GHz
  • 16 GB RAM
  • Advantech Co. Ltd Device M30 card and fpga.
  • Base OS: centos 7.6
  • OPenstack version queens
  • backend storage ceph

vScaler setup and passthrough

We have setup an ALL in one scaler setup to passthrough the advantech card. The globals files for the installation is attached. 

[root@localhost ~]# cat /etc/kolla/globals.yml | grep -v '#'
---
kolla_base_distro: "centos"
kolla_install_type: "binary"
openstack_release: "6.0.1"
node_custom_config: "/etc/kolla/config"
kolla_internal_vip_address: "185.93.28.189"
kolla_external_vip_address: "{{ kolla_internal_vip_address }}"
docker_registry: "registry.vscaler.com:5000"
network_interface: "eno1"
neutron_external_interface: "eno2"
enable_ceph: "yes"
enable_cinder: "yes"
enable_haproxy: "no"


The advantech card has 2 parts M30 codec and fpga part . check in lscpi -s 

[root@localhost ~]#lspci -s 09:00.0 
Multimedia controller: Advantech Co. Ltd Device 004b (rev 01)

[root@localhost ~]#lspci -s 0b:00.0 
0b:00.0 Multimedia controller: Device 50c1:0003 (rev 01)

Advantech cards pass-through works with the following set of QEMU* version of rpms. Download the same rpm to make the passthrough work

Note : the other Qemu* rpm will end the system in Hang mode and restart after creating virtual machine. 

[root@localhost ~]# ls qemu/
qemu-img-ev-2.6.0-28.el7.centos_3.6.1.x86_64.rpm         qemu-kvm-ev-2.6.0-28.el7.centos_3.6.1.x86_64.rpm
qemu-kvm-common-ev-2.6.0-28.el7.centos_3.6.1.x86_64.rpm  qemu-kvm-tools-ev-2.6.0-28.el7.centos_3.6.1.x86_64.rpm

root@localhost ~]# docker exec -it -u root nova_libvirt rpm  -qa | grep qemu
qemu-kvm-common-ev-2.10.0-21.el7_5.4.1.x86_64
qemu-kvm-ev-2.10.0-21.el7_5.4.1.x86_64
ipxe-roms-qemu-20170123-1.git4e85b27.el7_4.1.noarch
qemu-img-ev-2.10.0-21.el7_5.4.1.x86_64
libvirt-daemon-driver-qemu-3.9.0-14.el7_5.6.x86_64
centos-release-qemu-ev-1.0-3.el7.centos.noarch

[root@localhost ~]# docker exec -it -u root nova_compute rpm  -qa | grep qemu
centos-release-qemu-ev-1.0-3.el7.centos.noarch
ipxe-roms-qemu-20170123-1.git4e85b27.el7_4.1.noarch
qemu-kvm-common-ev-2.10.0-21.el7_5.4.1.x86_64
libvirt-daemon-driver-qemu-3.9.0-14.el7_5.6.x86_64
qemu-kvm-ev-2.10.0-21.el7_5.4.1.x86_64
qemu-img-ev-2.10.0-21.el7_5.4.1.x86_64


[root@localhost ~]# docker exec -it -u root glance_api rpm  -qa | grep qemu
qemu-img-ev-2.10.0-21.el7_5.4.1.x86_64
centos-release-qemu-ev-1.0-3.el7.centos.noarch

The containers with Qemu* and QEMU-kvm* rpms are nova_libvirt , nova_compute , glance_api . The changes were not needed in qemu-img-ev* in cinder* and ceph containers.

Configure nova-api (Controller)

Specify the PCI alias for the device.

Change the nova.conf libvirt are mandatory. Add the pci_white list information to while list the 


[root@localhost ~]# vi /etc/kolla/config/nova.conf
[root@localhost ~]# cat /etc/kolla/config/nova.conf
[filter_scheduler]
enabled_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, PciPassthroughFilter
available_filters=nova.scheduler.filters.all_filters

[libvirt]
cpu_mode = custom
cpu_model = Skylake-Client

[pci]
passthrough_whitelist = { "address": "0000:05:00.0" }
alias={"vendor_id":"50c1","product_id":"0003","device_type":"type-PCI","name":"a1"}

[root@localhost ~]# kolla-ansible -i all-in-one reconfigure
#check nova-scheduler , nova-api , nova-compute all have the setting

docker exec -it nova_compute cat /etc/nova/nova.conf
....
[filter_scheduler]
enabled_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, PciPassthroughFilter
available_filters = nova.scheduler.filters.all_filters

[pci]
passthrough_whitelist = { "address": "0000:05:00.0" }
alias = {"vendor_id":"50c1","product_id":"0003","device_type":"type-PCI","name":"a1"}

docker exec -it nova_api cat /etc/nova/nova.conf

....
[filter_scheduler]
enabled_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, PciPassthroughFilter
available_filters = nova.scheduler.filters.all_filters

[pci]
passthrough_whitelist = { "address": "0000:05:00.0" }
alias = {"vendor_id":"50c1","product_id":"0003","device_type":"type-PCI","name":"a1"}

docker exec -it nova_compute cat /etc/nova/nova.conf


== Reconfgigure the Kolla ansible setup to mreflect the changes 

[root@localhost ~]# kolla-ansible -i all_in_one reconfigure


[root@localhost ~]# openstack image create --disk-format raw --container-format bare --public  --file CentOS-7-x86_64-Everything-1804.iso Centos7.5_everything


# Make sure enough RAM is allocated 8G worked at time of testing 

[root@localhost ~]# openstack flavor create m1.pass_8G --ram 8024 --disk 0 --vcpu 2
[root@localhost ~]# openstack flavor set m1.pass_8G --property "pci_passthrough:alias"="a1:2"
[root@localhost ~]# openstack flavor show  m1.pass_8G
+----------------------------+--------------------------------------+
| Field                      | Value                                |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled   | False                                |
| OS-FLV-EXT-DATA:ephemeral  | 0                                    |
| access_project_ids         | None                                 |
| disk                       | 0                                    |
| id                         | d476fa2d-71d6-4686-b363-2b84baac3a3d |
| name                       | m1.pass_8G                           |
| os-flavor-access:is_public | True                                 |
| properties                 | pci_passthrough:alias='a1:1'         |
| ram                        | 8024                                 |
| rxtx_factor                | 1.0                                  |
| swap                       |                                      |
| vcpus                      | 2                                    |
+----------------------------+--------------------------------------+
[root@localhost ~]#
# to enable vfio drivers 

root@localhost ~]# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=50c1:0003
root@localhost ~]# echo 'vfio-pci' > /etc/modules-load.d/vfio-pci.conf
root@localhost ~]# reboot

# In base OS 
root@localhost ~]# lspci -vvvvvvns 05:00
[root@localhost ~]# lspci -vvns 05:00
05:00.0 0480: 50c1:0003 (rev 01)
        Subsystem: 50c1:0001
        Physical Slot: 41
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin ? routed to IRQ 66
        Region 0: Memory at df400000 (32-bit, non-prefetchable) [size=128K]
        Region 2: Memory at df200000 (32-bit, non-prefetchable) [size=2M]
        Region 4: Memory at df430000 (32-bit, non-prefetchable) [size=32K]
        Region 5: Memory at df420000 (32-bit, non-prefetchable) [size=64K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [c0] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000fee004d8  Data: 0000
        Capabilities: [80] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 1024 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 25.000W
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [200 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=01
                        Status: NegoPending- InProgress-
        Capabilities: [3a0 v1] Device Serial Number 00-00-00-00-00-00-00-01
        Capabilities: [400 v1] Vendor Specific Information: ID=0020 Rev=3 Len=100 <?>
        '''Kernel driver in use: vfio-pci'''


[root@localhost ~]# dmesg | grep 05:00
[    0.223047] pci 0000:05:00.0: [50c1:0003] type 00 class 0x048000
[    0.223063] pci 0000:05:00.0: reg 0x10: [mem 0xdf400000-0xdf41ffff]
[    0.223077] pci 0000:05:00.0: reg 0x18: [mem 0xdf200000-0xdf3fffff]
[    0.223091] pci 0000:05:00.0: reg 0x20: [mem 0xdf430000-0xdf437fff]
[    0.223099] pci 0000:05:00.0: reg 0x24: [mem 0xdf420000-0xdf42ffff]
[    0.223154] pci 0000:05:00.0: supports D1 D2
[    0.223155] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.515870] iommu: Adding device 0000:05:00.0 to group 1
[    0.548264] pci 0000:05:00.0: Signaling PME through PCIe PME interrupt
[ 2036.852908] vfio-pci 0000:05:00.0: enabling device (0400 -> 0402)
[12533.193714] vfio-pci 0000:05:00.0: enabling device (0400 -> 0402)
[12751.678451] vfio-pci 0000:05:00.0: irq 66 for MSI/MSI-X           #IRQ will be given Once the PCIe device is assigned in VM
[179033.494736] vfio-pci 0000:05:00.0: irq 66 for MSI/MSI-X
[root@mediaflow-bioslab scratch]# tar -xvf vega_bqb_release_1_4_2_14270_x64.tar.bz2
[root@mediaflow-bioslab scratch]# cd vega_bqb_release_1_4_2/vega3311_fpga_x64/
driver/       example/      install.log   install.sh    Makefile      sdk/          tools/        uninstall.sh
[root@mediaflow-bioslab vega3311_fpga_x64]# ./install.sh
Installation start.
CentOS Linux release 7.4.1708 (Core)
kernel-headers-3.10.0-693.el7.x86_64
kernel-devel-3.10.0-693.el7.x86_64
gcc-4.8.5-39.el7.x86_64
Making sample applications...
done.
Installing sample applications...
cp -f ./example/vega3311_capture_app/bin/vega3311_cap_app /usr/local/bin
cp -f ./example/vega3311_capture_config/bin/vega3311_cap_config /usr/local/bin
cp -f ./example/vega3311_transmit_config/bin/vega3311_trans_config /usr/local/bin
cp -f ./tools/vega3311_fpgautil/bin/vega3311_fpgautil /usr/local/bin
Installing header file and library...
mkdir -p /usr/local/include/libvega_capture_api
mkdir -p /usr/local/include/libvega_transmit_api
cp -f ./driver/lib/libvega3311_fpga_drv.a /usr/local/lib
cp -f ./driver/lib/libvega3311_fpga_drv.so /usr/local/lib
cp -f ./sdk/vega3311_capture_api/lib/libvega3311_capture.a /usr/local/lib
cp -f ./sdk/vega3311_capture_api/lib/libvega3311_capture.so /usr/local/lib
cp -f ./sdk/vega3311_transmit_api/lib/libvega3311_transmit.a /usr/local/lib
cp -f ./sdk/vega3311_transmit_api/lib/libvega3311_transmit.so /usr/local/lib
cp -f ./driver/include/vega3311_fpga_drv.h /usr/local/include/libvega_capture_api
cp -f ./sdk/vega3311_capture_api/include/VEGA3311_capture.h /usr/local/include/libvega_capture_api
cp -f ./sdk/vega3311_capture_api/include/VEGA3311_cap_types.h /usr/local/include/libvega_capture_api
cp -f ./sdk/vega3311_transmit_api/include/VEGA3311_transmit.h /usr/local/include/libvega_transmit_api
cp -f ./sdk/vega3311_transmit_api/include/VEGA3311_trans_types.h /usr/local/include/libvega_transmit_api
Clean the package...
done.
Making and loading device driver...done.
Installation completed.

[root@mediaflow-bioslab scratch]# lspci -vs 09:00.0
09:00.0 Multimedia controller: Advantech Co. Ltd Device 004b (rev 01)
	Subsystem: Advantech Co. Ltd Device 004b
	Physical Slot: 1
	Flags: bus master, fast devsel, latency 0, IRQ 18
	Memory at de800000 (32-bit, non-prefetchable) [size=128K]
	Memory at de400000 (32-bit, non-prefetchable) [size=4M]
	Memory at 3fff800000 (64-bit, prefetchable) [size=4M]
	Capabilities: [80] Power Management version 3
	Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [b0] MSI-X: Enable+ Count=32 Masked-
	Capabilities: [c0] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [340] Vendor Specific Information: ID=0001 Rev=0 Len=02c <?>
	Kernel driver in use: vega3311_fpga
	Kernel modules: vega3311_fpga

To verify the drivers are installed correctly chyeck the following command 

[root@mediaflow-bioslab scratch]# vega3311_cap_app list_dev
DRV INF: SF: Got idcode 0xc2 0x20 0x1a
Detected 1 chip(s).
==============================================
Chip: 0
	Device node name: /dev/vega3311_fpga_0
	PCI Bridge 0x08
	PCI Bus:Slot.Func 09:00.0
	Vendor ID: 0x13fe	Device ID: 0x004b
	Subsystem VendorID: 0x13fe	Subsystem DevID: 0x004b
	Serial Number: LKD0114896	Board ID: 9792A33111E
	Irq: 18
	BAR Configuration:
	Start              | Length
	0x00000000de800000 | 000131072
	0x00000000de400000 | 004194304
==============================================
[root@mediaflow-bioslab scratch]# 
[root@mediaflow-bioslab scratch]# vega3311_fpgautil list
	[Card 0] Device id 0, PCIe bridge 8

Compling drivers for the fgpa (history back up from Roy VM) 

1  sudo su -
    2  exit
    3  lspci
    4  lspci -vvs 05.0
    5  lspci -vvvs 05.0
    6  sudo lspci -vvvs 05.0
    7  sudo lspci -vvvs 06.0
    8  ls
    9  tar -jxvf vega_bqb_trc_release_1_5_1_12817_x64.tar.bz2
   10  sudo yum list bzip2
   11  sudo yum install bzip2
   12  tar -jxvf vega_bqb_trc_release_1_5_1_12817_x64.tar.bz2
   13  ls
   14  cd vega_bqb_trc_release_1_5_1
   15  ls
   16  cd vega_bqb_codec_x64/01_HOST/realtime_codec_soft
   17  ls
   18  sudo ./install.sh
   19  uname -r
   20  sudo yum install kernel-headers-3.10.0-862.2.3.el7.x86_64
   21  sudo yum install kernel-headers-3.10.0-862.2.3
   22  sudo yum list kernel-headers
   23  sudo yum list kernel-headers --showduplicates
   24  uname -r
   25  sudo yum update
   26  sudo systemctl reboot
   27  uname -r
   28  cd vega_bqb_trc_release_1_5_1
   29  cd vega_bqb_codec_x64/01_HOST/realtime_codec_soft
   30  sudo ./install.sh
   31  sudo dmesg | grep m30trc
   32  vega_bqb_vtrc version
   33  ls /dev
   34  lsmob
   35  lsmod
   36  ls
   37  sudo ./uninstall.sh
   38  ls
   39  lsmod
   40  cd 
   41  #cd ~
   42  ls
   43  rm -r vega_bqb_trc_release_1_5_1
   44  ls
   45  rm vega_bqb_trc_release_1_5_1_12817_x64.tar.bz2
   46  ls
   47  tar -jxvf vega_bqb_release_1_3_1_12729_x64.tar.bz2
   48  ls
   49  cd vega_bqb_release_1_3_1
   50  cd vega_bqb_codec_x64/01_HOST/realtime_codec_soft/
   51  sudo ./install.sh
   52  sudo dmesg | grep "m30*"
   53  cd ~
   54  cd vega_bqb_release_1_3_1
   55  cd vega3311_fpga_x64/
   56  ls
   57  sudo ./install.sh
   58  sudo dmesg | grep "vega3311*"
   59  vega_bqb_sysctl set_enc --device 0
   60  vega_bqb_sysctl version
   61  vega_bqb_sysctl info
   62  vega_bqb_venc version
   63  sudo apt-get install mercurial cmake cmake-curses-gui build-essential yasm
   64  sudo yum install mercurial cmake cmake-gui
   65  sudo yum groupinstall "Development Tools"
   66  sudo yum install -y epel-release
   67  sudo yum --disablerepo="*" --enablerepo="epel" install -y yasm
   68  sudo yum install -y SDL-devel
   69  cd ~
   70  wget http://sourceforge.net/projects/boost/files/boost/1.54.0/boost_1_54_0.tar.gz
   71  sudo yum install wget
   72  wget http://sourceforge.net/projects/boost/files/boost/1.54.0/boost_1_54_0.tar.gz
   73  tar -xzvf boost_1_54_0.tar.gz
   74  cd ~/boost_1_54_0
   75  ./bootstrap.sh
   76  sudo ./bjam --layout=system --toolset=gcc install
   77  cd ~/vega_bqb_release_1_3_1/ffmpeg-3.4.1/vega_bqb_avc/build/linux
   78  PATH="/usr/local/bin:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="/usr/local" -DENABLE_SHARED:bool=on -DENABLE_PIC:bool=on ../../source
   79  make
   80  sudo make install
   81  cd ~/vega_bqb_release_1_3_1/ffmpeg-3.4.1/vega_bqb_hevc/build/linux
   82  PATH="/usr/local/bin:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="/usr/local" -DENABLE_SHARED:bool=on -DENABLE_PIC:bool=on ../../source
   83  make
   84  sudo make install
   85  cd ~/vega_bqb_release_1_3_1/ffmpeg-3.4.1/vega3311_ff_capture/build/linux
   86  PATH="/usr/local/bin:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="/usr/local" -DENABLE_SHARED:bool=on -DENABLE_PIC:bool=on ../../source
   87  make
   88  sudo make install
   89  cd ~/vega_bqb_release_1_3_1/ffmpeg-3.4.1/ffmpeg_source
   90  ls /usr/local/lib
   91  ls /usr/local/lib/pkgconfig
   92  PATH="/usr/local/bin:$PATH" PKG_CONFIG_PATH="/usr/local/lib/pkgconfig"
   93  ./configure --enable-shared --enable-libvegabqbavc --enable-libvegabqbhevc --enable-nonfree
   94  echo $PKG_CONFIG_PATH
   95  pkg-config --help
   96  pkg-config --exists --print-errors libvegabqbavc
   97  find /usr -name libvegabqbavc.pc
   98  find /usr -name vega_bqb_hevc.pc
   99  find ~/ -name libvegabqbavc.pc
  100  find ~/ -name vega_bqb_hevc.pc
  101  less /home/centos/vega_bqb_release_1_3_1/ffmpeg-3.4.1/vega_bqb_hevc/build/linux/vega_bqb_hevc.pc
  102  make
  103  sudo make install
  104  cd ~
  105  sudo ldconfig
  106  ffmpeg
  107  ls mnt
  108  ls /mnt
  109  ls /media
  110  df -h
  111  lspci -t
  112  lspci -vmms 05.0
  113  lspci -vmms 06.0
  114  vega3311_cap_app qsrc 0 0
  115  vega3311_cap_app qsrc 0 1
  116  vega3311_cap_app qsrc 0 2
  117  vega3311_cap_app qsrc 0 3
  118  vega3311_cap_app qsrc 1 0
  119  vega3311_cap_config 0 0 -input_mode 4ch_fhd
  120  vega3311_cap_config 0 0 -input_mode 4ch
  121  vega3311_cap_app qsrc 0 0
  122  vega3311_cap_app qsrc 0 1
  123  vega3311_cap_config 0 0 -image_format nv12
  124  vega3311_cap_config -h
  125  vega3311_cap_app -h
  126  vega3311_cap_app reset 0 0
  127  vega3311_cap_app list_dev
  128  systemctl reboot
  129  sudo systemctl reboot
  130  vega3311_cap_config 0 0 -image_format nv16
  131  ls
  132  cd vega_bqb_release_1_3_1
  133  ls
  134  cd vega3311_fpga_x64/tools/fpga_fw/
  135  ls
  136  vega3311_cap_app qsrc 0 0
  137  vega_bqb_venc version
  138  vega_bqb_sysctl set_dec --device 0
  139  vega_bqb_venc version
  140  ls
  141  cd ..
  142  ls
  143  cd vega_bqb_codec_x64
  144  ls
  145  cd 02_M30
  146  ls
  147  cd 02_tools
  148  ls
  149  mv config_system_dec.txt ../
  150  ls
  151  sudo ./flash_write_all.sh 0
  152  sudo systemctl reboot
  153  vega_bqb_venc version
  154  sudo lspci -vvvs 05.0
  155  sudo lspci -vvvvs 05.0
  156  sudo lspci -vvvvs 06.0
  157  ls
  158  cd vega_bqb_release_1_3_1
  159  ls
  160  cd vega3311_fpga_x64/tools/fpga_fw
  161  ls
  162  cd ..
  163  ls
  164  cd vega3311_fpgautil
  165  ls
  166  cd ..
  167  cd vega3311_fpga_x64/tools/fpga_fw
  168  ls
  169  vega3311_fpgautil fw_update 0 vega3311_0_6C_20181029_3G12G_wt.bin
  170  lspci
  171  lspci -m
  172  lspci -mm
  173  lspci -v
  174  free -h
  175  vega3311_cap_app qsrc 0 0
  176  vega3311_cap_app list_dev
  177  sudo dmesg | grep -e BAR
  178  sudo dmesg | grep -e vega3311_fpga
  179  sudo lspci -vvvs 05.0
  180  ls /sys
  181  ls /sys/dev
  182  ls /sys/device
  183  sudo ls /sys/device
  184  sudo ls /sys/devices
  185  sudo ls /sys/bus
  186  ls /sys/bus
  187  ls /sys/bus/pci
  188  ls /sys/bus/pci/devices
  189  ls /sys/bus/pci/devices/0000:00:05.0
  190  ls /sys/bus/pci/devices/0000:00:05.0/vega3311_fpga
  191  ls /sys/bus/pci/devices/0000:00:05.0/vega3311_fpga/vega3311_fpga_0
  192  ls /sys/bus/pci/devices/0000:00:05.0
  193  ls -l /sys/bus/pci/devices/0000:00:05.0
  194  ls -l /sys/bus/pci/devices/0000:00:05.0/
  195  sudo less -l /sys/bus/pci/devices/0000:00:05.0/max_link_width
  196  sudo less -l /sys/bus/pci/devices/0000:00:05.0/max_link_speed
  197  sudo less -l /sys/bus/pci/devices/0000:00:05.0/irq
  198  ls -l /sys/bus/pci/devices/0000:00:05.0/firmware_node
  199  ls -l /sys/bus/pci/devices/0000:00:05.0/firmware_node/
  200  sudo less /sys/bus/pci/devices/0000:00:05.0/firmware_node/adr
  201  cd /~
  202  cd ~
  203  ls
  204  vega_bqb_sysctl set_enc –device 0
  205  vega_bqb_sysctl set_enc --device 0
  206  vega_bqb_venc version
  207  vega_bqb_venc reset
  208  ls /dev
  209  sudo systemctl reboot
  210  vega_bqb_venc version
  211  sudo systemctl poweroff
  212  vega_bqb_venc version
  213  vega_bqb_sysctl set_enc --device 0
  214  vega_bqb_venc version
  215  vega_bqb_sysctl set_dec --device 0
  216  vega_bqb_venc version
  217  ls
  218  cd vega_bqb_release_1_3_1
  219  cd vega_bqb_codec_x64/02_M30
  220  ls
  221  mv config_system_dec.txt ./01_M30_firm  02_tools
  222  ls
  223  ls 02_tools
  224  cd 02_tools
  225  ls
  226  mv 01_M30_firm ../
  227  ls
  228  ls ../
  229  mv config_system.txt ../
  230  ls
  231  sudo ./flash_write_all.sh 0
  232  ls
  233  less flash_write_mcpu_fw.sh
  234  sudo systemctl reboot
  235  vega_bqb_sysctl set_enc --device 0
  236  vega_bqb_venc version
  237  vega_bqb_sysctl set_dec --device 0
  238  vega_bqb_venc version
  239  ls
  240  cd vega_bqb_release_1_3_1
  241  ls
  242  cd vega_bqb_codec_x64/02_M30
  243  ls
  244  mv config_system.txt ./02_tools
  245  ls
  246  cd 02_tools
  247  ls
  248  sudo ./flash_write_all.sh 0
  249  sudo poweroff
  250  vega_bqb_sysctl set_enc --device 0
  251  vega_bqb_venc version
  252  vega3311_cap_app list_dev
  253  logout
  254  yum list tmux
  255  yum list screen
  256  vega3311_cap_app list_dev
  257  sudo vega3311_cap_app list_dev
  258  sudo /usr/local/bin/vega3311_cap_app list_dev
  259  lsmod
  260  free -h
  261  df -h
  262  ls
  263  ffmpeg -vsync vfr -f mpegts -i ~/Videos/bbb.ts -c:v libvegabqbavc -b:v 4000k -vegabqbavc-params profile=high:level-idc=51:open-gop=2:keyint=30:scenecut=1:bframes=2:deblock=1:rate-ctrl=cbr:bitrate=4000:sar=1:device=0:channel=0:input-mode=0 -c:a aac ~/Videos/bbb.mp4
  264  alias
  265  logout
  266  lspci
  267  sudo lspci -vvv -s 05.0
  268  sudo poweroff
  269  lspci
  270  sudo lspci -vvv -s 05.0
  271  vega3311_cap_app qsrc 0 0
  272  vega_bqb_sysctl set_enc --device 0
  273  vega_bqb_venc version
  274  vega_bqb_sysctl set_dec --device 0
  275  vega3311_cap_app list_dev
  276  sudo lspci -vvv -s 05.0
  277  ls
  278  cd vega_bqb_release_1_3_1
  279  ls
  280  cd vega3311_fpga_x64
  281  ls
  282  cd ../vega3310_fpga_x64
  283  ls
  284  cd ..
  285  cd vega3311_fpga_x64
  286  less install.log
  287  modinfo vega3311_fpga
  288  lspci
  289  lspci -n -s 05.0
  290  sudo lspci -vvvv -s 5.0
  291  ls /sys/bus/pci
  292  ls /sys/bus/pci/devices
  293  ls /sys/bus/pci/devices/0000:00:05.0
  294  ls -l
  295  ls -l /sys/bus/pci/devices/0000:00:05.0/
  296  ls -l /sys/bus/pci/devices/0000:00:06.0/
  297  ls -l /sys/bus/pci/devices/0000:00:05.0/firmware_node/
  298  less /sys/bus/pci/devices/0000:00:05.0/irq
  299  less /sys/bus/pci/devices/0000:00:06.0/irq
  300  sudo cat /sys/bus/pci/devices/0000:00:05.0/irq
  301  sudo cat /sys/bus/pci/devices/0000:00:06.0/irq
  302  vega_bqb_sysctl set_enc --device 0
  303  vega_bqb_venc version
  304  vega_bqb_venc version
  305  stty -ixon
  306  sudo lspci -vvv -s 05.0
  307  sudo cat /proc/interrupts
  308  sudo lspci -vvv -s 06.0
  309  logout
  310  sudo cat /proc/interrupts
  311  vega3311_cap_app qsrc 0 0
  312  sudo cat /proc/interrupts
  313  sudo journalctl -k | grep remap
  314  sudo poweroff
  315  sudo cat /proc/interrupts
  316  vega3311_cap_app qsrc 0 0
  317  sudo cat /proc/interrupts
  318  sudo dmesg | grep less
  319  sudo dmesg | less
  320  sudo cat /proc/interrupts
  321  sudo dmesg | less
  322  sudo poweroff
  323  sudo dmesg | less
  324  ls
  325  cd vega_bqb_release_1_3_1
  326  ls
  327  cd vega3311_fpga_x64
  328  ls
  329  less install.log
  330  sudo ./uninstall.sh
  331  lsmod
  332  sudo dmesg | less
  333  sudo cat /proc/interrupts
  334  sudo lspci -vvv -s 05.0
  335  ls
  336  sudo ./uninstall.sh
  337  sudo ./install.sh
  338  sudo dmesg | less
  339  sudo cat /proc/interrupts
  340  vega3311_cap_app list_dev
  341  sudo cat /proc/interrupts
  342  sudo dmesg | less
  343  sudo cat /proc/interrupts
  344  vega3311_cap_app list_dev
  345  sudo cat /proc/interrupts
  346  vega3311_cap_app list_dev
  347  sudo cat /proc/interrupts
  348  vega_bqb_sysctl set_enc --device 0
  349  sudo cat /proc/interrupts
  350  vega_bqb_venc version
  351  sudo cat /proc/interrupts
  352  ls -l /sys/bus/pci/devices/0000:00:05.0/
  353  cat /sys/bus/pci/devices/0000:00:05.0/enable
  354  cat /sys/bus/pci/devices/0000:00:05.0/msi_bus
  355  ls -l /sys/bus/pci/devices/0000:00:05.0/msi_irqs/
  356  cat /sys/bus/pci/devices/0000:00:05.0/msi_irqs/29
  357  cat /sys/bus/pci/devices/0000:00:05.0/msi_irqs/30
  358  cat /sys/bus/pci/devices/0000:00:05.0/msi_irqs/37
  359  ls
  360  ffmpeg
  361  logout
  362  ls
  363  cd vega_bqb_release_1_3_1
  364  ls
  365  cd vega3311_ff_capture/build/linux
  366  ls
  367  cd ffmpeg-3.4.1
  368  cd vega3311_ff_capture/build/linux
  369  ls
  370  PATH="/usr/local/bin:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="/usr/local" -DENABLE_SHARED:bool=on -DENABLE_PIC:bool=on ../../source
  371  make
  372  sudo make install
  373  cd ~/vega_bqb_release_1_3_1
  374  cd ffmpeg-3.4.1
  375  cd ffmpeg_source
  376  ls
  377  PATH="/usr/local/bin:$PATH" PKG_CONFIG_PATH="/usr/local/lib/pkgconfig"
  378  ./configure --enable-shared --enable-libvegabqbavc --enable-libvegabqbhevc --enable-libvega3311-capture --enable-nonfree
  379  logout
  380  dmesg | grep -i 05.0
  381  sudo cat /proc/interrupts
  382  sudo poweroff
  383  sudo cat /proc/interrupts
  384  lspci -vs 05.0
  385  sudolspci -vs 05.0
  386  sudo lspci -vs 05.0
  387  sudo lspci -vv | grep -i "IRQ 10"
  388  sudo lspci -vs 06.0
  389  sudo lspci -vv | grep -i "IRQ 50"
  390  logout
  391  sudo su -
  392  ls
  393  vi Videos/
  394  cd Videos/
  395  ls
  396  cd ..
  397  ls
  398  history
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=VScaler:_Advantech_PCIe_passthrough&oldid=29804"