Binary data analysis and visualization tool

Veles - A new age tool for binary analysis

A .tar file as seen in Veles

It is a very difficult task for a human to notice subtle patterns in large amounts of binary data, however, us humans are very good at finding patterns in images. Statistical visualizations let you find the important bits in a sea of binary data - all at a glance.

See our home page at https://veles.io or visit us on IRC: #veles at freenode.

Binaries

You can download compiled binaries at https://veles.io or https://github.com/codilime/veles/releases.

Building

See BUILDING.md for detailed instructions.

Contributing

See CONTRIBUTING.md.

Comments
  • Specific usage CMAKE_OSX_DEPLOYMENT_TARGET

    Specific usage CMAKE_OSX_DEPLOYMENT_TARGET

  • Add keyboard shortcuts editor

    Add keyboard shortcuts editor

    Keyboard shortcuts for all action types are saved in settings, they are editable through new dialog available in file menu. All existing QActions that weren't generated based on some data were converted to support editing shortcuts for them.

  • Wip/ji/dbif

    Wip/ji/dbif

    It's possible now to open files and manipulate chunks from multiple clients. No parsers working yet and there are still things to improve, but it may be a good moment to have a look (or even merge).

  • Slot mismatch in QObject::connect in VisualizationPanel

    Slot mismatch in QObject::connect in VisualizationPanel

    When starting 3d visualization on 7d973b6779374d45dc415fd24e27652aaa0b8f85 on x86_64-linux:

    QObject::connect: No such slot veles::visualization::VisualizationPanel::setSampleSize(int)
    QObject::connect: (sender name: 'SamplingMethodDialog')
    QObject::connect: Incompatible sender/receiver arguments
    QCheckBox::stateChanged(int) --> veles::visualization::TrigramWidget::setUseBrightnessHeuristic(Qt::CheckState)

  • Graphicas card requirement?

    Graphicas card requirement?

    Just wanna double check if veles requires nvida graphics card. I installed the .dev on ubuntu smoothly but was not able to run it. See the following output.

    ~$ veles
    libGL error: No matching fbConfigs or visuals found
    libGL error: failed to load driver: swrast
    Unrecognized OpenGL version
    Unrecognized OpenGL version
    
  • New readme

    New readme

    I refreshed our readme a bit by splitting it into files and adding more detailed build instructions. I also converted everything into MD format, because it seems that GitHub's implementation of RST can't highlight code.

    I'm also trying to find a good screen for the main readme, currently I have two candidates:

    • https://github.com/codilime/veles/tree/9bb8edbd6624b7cde07194a2d9d9aba015acdc40
    • https://github.com/codilime/veles/tree/e7324b1d6a8f6ba9c23f40cf41e7b2d8e53c7e46

    If you have an idea for a better screenshot please go ahead and paste it in the comments :)

    Fixes #360.

  • CMake complains about cyclic dependencies

    CMake complains about cyclic dependencies

    I did a git checkout 2017.06.0.SWOND cmake -D CMAKE_BUILD_TYPE=Release .. gives

    CMake Error: The inter-target dependency graph contains the following strongly connected component (cycle):
      "parser" of type STATIC_LIBRARY
        depends on "veles_db" (weak)
        depends on "parser_autogen" (strong)
      "veles_db" of type STATIC_LIBRARY
        depends on "parser" (weak)
        depends on "veles_db_autogen" (strong)
      "parser_autogen" of type UTILITY
        depends on "veles_db" (strong)
      "veles_db_autogen" of type UTILITY
        depends on "parser" (strong)
    At least one of these targets is not a STATIC_LIBRARY.  Cyclic dependencies are allowed only among static libraries.
    
  • Debian pls!?

    Debian pls!?

    Hi, is possible to get veles to run on debian testing(or)stretch ? the issue is that in debian, python2 is default by nature & py3 req.txt are all based on py3.

    thx in adv.

  • NetworkClient: Disconnect. error

    NetworkClient: Disconnect. error

    I am using archlinux and installed veles using AUR But when i am trying to run veles i got this error?

    Waiting for a new server to start...
    Process of locally created server started.
    INFO:root:Świtezianka server is starting up...
    INFO:root:Opening database...
    Traceback (most recent call last):
      File "srv.py", line 51, in <module>
        conn = AsyncLocalConnection(loop, args.database)
      File "/usr/lib/python3.8/site-packages/veles/server/conn.py", line 78, in __init__
        tracker = DbTracker(tracker)
      File "/usr/lib/python3.8/site-packages/veles/db/tracker.py", line 49, in __init__
        db = DbBackend(db)
      File "/usr/lib/python3.8/site-packages/veles/db/backend.py", line 135, in __init__
        wrapper = msgpackwrap.MsgpackWrapper()
      File "/usr/lib/python3.8/site-packages/veles/proto/msgpackwrap.py", line 34, in __init__
        self.unpacker = msgpack.Unpacker(
      File "msgpack/_unpacker.pyx", line 317, in msgpack._cmsgpack.Unpacker.__init__
    TypeError: __init__() got an unexpected keyword argument 'encoding'
    Process of locally created server finished. Exit code: 1.
    NetworkClient: Disconnect.
    

    Installed requirements are:

    python-msgpack 1.0.0-1 [installed] python-pbr 5.4.5-1 [installed] python-six 1.15.0-1 [installed] python-pyopenssl 19.1.0-2 [installed]

  • install of Veles_2018.05_64bit_Ubuntu1604.deb fails with error: invalid command 'bdist_wheel'

    install of Veles_2018.05_64bit_Ubuntu1604.deb fails with error: invalid command 'bdist_wheel'

    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    Reading state information... Done
    
    Veles tool for binary data analysis
    Do you want to install the software package? [y/N]:y
    /usr/bin/gdebi:113: FutureWarning: Possible nested set at position 1
      c = findall("[[(](\S+)/\S+[])]", msg)[0].lower()
    (Reading database ... 486103 files and directories currently installed.)
    Preparing to unpack Veles_2018.05_64bit_Ubuntu1604.deb ...
    Unpacking veles (2018.05) over (2018.05) ...
    Setting up veles (2018.05) ...
    Collecting pbr>=1.3 (from -r /usr/share/veles-server/requirements.txt (line 1))
      Using cached https://files.pythonhosted.org/packages/7a/db/a968fd7beb9fe06901c1841cb25c9ccb666ca1b9a19b114d1bbedf1126fc/pbr-5.4.4-py2.py3-none-any.whl
    Collecting six==1.10.0 (from -r /usr/share/veles-server/requirements.txt (line 2))
      Using cached https://files.pythonhosted.org/packages/c8/0a/b6723e1bc4c516cb687841499455a8505b44607ab535be01091c0f24f079/six-1.10.0-py2.py3-none-any.whl
    Collecting msgpack-python==0.4.8 (from -r /usr/share/veles-server/requirements.txt (line 3))
      Using cached https://files.pythonhosted.org/packages/21/27/8a1d82041c7a2a51fcc73675875a5f9ea06c2663e02fcfeb708be1d081a0/msgpack-python-0.4.8.tar.gz
    Collecting pyOpenSSL (from -r /usr/share/veles-server/requirements.txt (line 4))
      Using cached https://files.pythonhosted.org/packages/9e/de/f8342b68fa9e981d348039954657bdf681b2ab93de27443be51865ffa310/pyOpenSSL-19.1.0-py2.py3-none-any.whl
    Collecting cryptography>=2.8 (from pyOpenSSL->-r /usr/share/veles-server/requirements.txt (line 4))
      Using cached https://files.pythonhosted.org/packages/45/73/d18a8884de8bffdcda475728008b5b13be7fbef40a2acc81a0d5d524175d/cryptography-2.8-cp34-abi3-manylinux1_x86_64.whl
    Collecting cffi!=1.11.3,>=1.8 (from cryptography>=2.8->pyOpenSSL->-r /usr/share/veles-server/requirements.txt (line 4))
      Using cached https://files.pythonhosted.org/packages/16/cd/1f4ddf6be8300713c676bb9f3a2d3b8eb8accc0a6a24f57d4f6c4cd59d34/cffi-1.13.2-cp37-cp37m-manylinux1_x86_64.whl
    Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography>=2.8->pyOpenSSL->-r /usr/share/veles-server/requirements.txt (line 4))
      Using cached https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz
    Building wheels for collected packages: msgpack-python, pycparser
      Running setup.py bdist_wheel for msgpack-python ... error
      Complete output from command /usr/share/veles-server/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-u79zklvc/msgpack-python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-_nmpkf3m --python-tag cp37:
      usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
         or: -c --help [cmd1 cmd2 ...]
         or: -c --help-commands
         or: -c cmd --help
    
      error: invalid command 'bdist_wheel'
    
      ----------------------------------------
      Failed building wheel for msgpack-python
      Running setup.py clean for msgpack-python
      Running setup.py bdist_wheel for pycparser ... error
      Complete output from command /usr/share/veles-server/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-u79zklvc/pycparser/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-sg5sn7gd --python-tag cp37:
      usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
         or: -c --help [cmd1 cmd2 ...]
         or: -c --help-commands
         or: -c cmd --help
    
      error: invalid command 'bdist_wheel'
    
      ----------------------------------------
      Failed building wheel for pycparser
      Running setup.py clean for pycparser
    Failed to build msgpack-python pycparser
    Installing collected packages: pbr, six, msgpack-python, pycparser, cffi, cryptography, pyOpenSSL
      Running setup.py install for msgpack-python ... done
      Running setup.py install for pycparser ... done
    Successfully installed cffi-1.13.2 cryptography-2.8 msgpack-python-0.4.8 pbr-5.4.4 pyOpenSSL-19.1.0 pycparser-2.19 six-1.10.0
    Processing /usr/share/veles-server/veles-0.0.1.dev635.tar.gz
    Building wheels for collected packages: veles
      Running setup.py bdist_wheel for veles ... error
      Complete output from command /usr/share/veles-server/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-izxp_3w3/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-z5_1ocl8 --python-tag cp37:
      usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
         or: -c --help [cmd1 cmd2 ...]
         or: -c --help-commands
         or: -c cmd --help
    
      error: invalid command 'bdist_wheel'
    
      ----------------------------------------
      Failed building wheel for veles
      Running setup.py clean for veles
    Failed to build veles
    Installing collected packages: veles
      Running setup.py install for veles ... done
    Successfully installed veles-0.0.1.dev635
    Processing triggers for mime-support (3.64) ...
    Processing triggers for desktop-file-utils (0.24-1) ...
    

    This is on buster, python3-wheel and python-wheel are installed...

  • missing Kaitai Struct (.ksy) source for file format parsers

    missing Kaitai Struct (.ksy) source for file format parsers

    Please publish the .ksy files for the file format parsers, delete the following generated code from the repository and change the cmake files to always run kaitai-struct-compiler at build time to generate the C parser code.

    veles (master=) $ find -iname *.ksy
    veles (master=) $ grep -A1 generated include/kaitai/* src/kaitai/*
    include/kaitai/avi.h:// This is a generated file! Please edit source .ksy file and use
    include/kaitai/avi.h-// kaitai-struct-compiler to rebuild
    --
    include/kaitai/bmp.h:// This is a generated file! Please edit source .ksy file and use
    include/kaitai/bmp.h-// kaitai-struct-compiler to rebuild
    --
    include/kaitai/elf.h:// This is a generated file! Please edit source .ksy file and use
    include/kaitai/elf.h-// kaitai-struct-compiler to rebuild
    --
    include/kaitai/gif.h:// This is a generated file! Please edit source .ksy file and use
    include/kaitai/gif.h-// kaitai-struct-compiler to rebuild
    --
    include/kaitai/microsoft_pe.h:// This is a generated file! Please edit source .ksy file and use
    include/kaitai/microsoft_pe.h-// kaitai-struct-compiler to rebuild
    --
    include/kaitai/png.h:// This is a generated file! Please edit source .ksy file and use
    include/kaitai/png.h-// kaitai-struct-compiler to rebuild
    --
    include/kaitai/quicktime_mov.h:// This is a generated file! Please edit source .ksy file and use
    include/kaitai/quicktime_mov.h-// kaitai-struct-compiler to rebuild
    --
    include/kaitai/zip.h:// This is a generated file! Please edit source .ksy file and use
    include/kaitai/zip.h-// kaitai-struct-compiler to rebuild
    --
    src/kaitai/avi.cc:// This is a generated file! Please edit source .ksy file and use
    src/kaitai/avi.cc-// kaitai-struct-compiler to rebuild
    --
    src/kaitai/bmp.cc:// This is a generated file! Please edit source .ksy file and use
    src/kaitai/bmp.cc-// kaitai-struct-compiler to rebuild
    --
    src/kaitai/elf.cc:// This is a generated file! Please edit source .ksy file and use
    src/kaitai/elf.cc-// kaitai-struct-compiler to rebuild
    --
    src/kaitai/gif.cc:// This is a generated file! Please edit source .ksy file and use
    src/kaitai/gif.cc-// kaitai-struct-compiler to rebuild
    --
    src/kaitai/microsoft_pe.cc:// This is a generated file! Please edit source .ksy file and use
    src/kaitai/microsoft_pe.cc-// kaitai-struct-compiler to rebuild
    --
    src/kaitai/png.cc:// This is a generated file! Please edit source .ksy file and use
    src/kaitai/png.cc-// kaitai-struct-compiler to rebuild
    --
    src/kaitai/quicktime_mov.cc:// This is a generated file! Please edit source .ksy file and use
    src/kaitai/quicktime_mov.cc-// kaitai-struct-compiler to rebuild
    --
    src/kaitai/zip.cc:// This is a generated file! Please edit source .ksy file and use
    src/kaitai/zip.cc-// kaitai-struct-compiler to rebuild
    
  • Radare2 integration

    Radare2 integration

    Radare2 is a binary analysis and reverse engineering framework, that has disassemblers, analyzers, emulation, debuggers, diffing engine, etc. So it might be beneficial for Veles to do some kind of integration, at the very least for analyzing the code parts, magic search and emulation-based data visualization (to be able to visualize the data structures that are accessible only in RAM, e.g. after running some piece of code).

  • ImportError: No module named 'veles' on macOS

    ImportError: No module named 'veles' on macOS

    Latest veles for macOS (it reports itself as 2018.05.0.TIF in it's about dialogue) fails to launch properly on my system. The installed app bundle starts, but fails to do anything useful after showing the following error message:

    Trying to start a new server...
        working directory: /Applications/veles.app/Contents/Resources/veles-server
        python script name: srv.py
        python interpreter executable: /Applications/veles.app/Contents/MacOS/../Resources/veles-server/venv/bin/python3
        arguments:
            srv.py
            --cert-dir
            /Users/jdoe/Library/Application Support/CodiSec/Veles
            veles+ssl://[email protected]:3135
            /Users/jdoe/Library/Application Support/CodiSec/Veles/veles.vdb
    
    Waiting for a new server to start...
    Process of locally created server started.
    Traceback (most recent call last):
      File "srv.py", line 22, in <module>
        from veles.server.conn import AsyncLocalConnection
    ImportError: No module named 'veles'
    Process of locally created server finished. Exit code: 1.
    NetworkClient: Disconnect.
    

    This is macOS Sierra 10.12.6, uname is Darwin [redacted] 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT 2018; root:xnu-3789.73.14~1/RELEASE_X86_64 x86_64.

An easy to build CO2 Monitor/Meter with Android and iOS App for real time visualization and charting of air data, data logger, a variety of communication options (BLE, WIFI, MQTT, ESP-Now) and many supported sensors.
An easy to build CO2 Monitor/Meter with Android and iOS App for real time visualization and charting of air data, data logger, a variety of communication options (BLE, WIFI, MQTT, ESP-Now) and many supported sensors.

CO2-Gadget An easy to build CO2 Monitor/Meter with cell phone App for real time visualization and charting of air data, datalogger, a variety of commu

Dec 17, 2022
Maat is an open-source Dynamic Symbolic Execution and Binary Analysis framework
Maat is an open-source Dynamic Symbolic Execution and Binary Analysis framework

About Maat is an open-source Dynamic Symbolic Execution and Binary Analysis framework. It provides various functionalities such as symbolic execution,

Dec 27, 2022
This is a tool for software engineers to view,record and analyse data(sensor data and module data) In the process of software development.
This is a tool for software engineers to view,record and analyse data(sensor data and module data) In the process of software development.

![Contributors][Huang Jianyu] Statement 由于工具源码在网上公开,除使用部分开源项目代码外,其余代码均来自我个人,工具本身不包含公司的知识产权,所有与公司有关的内容均从软件包中移除,软件发布遵循Apache协议,任何人均可下载进行修改使用,如使用过程中出现任何问

Dec 25, 2022
The Time Series Visualization Tool that you deserve.
The Time Series Visualization Tool that you deserve.

Gold Sponsor: Greenzie PlotJuggler 3.3 PlotJuggler is a tool to visualize time series that is fast, powerful and intuitive. Noteworthy features: Simpl

Jan 8, 2023
Pyramid is a free, open GUI tool for offline shader validation and analysis
Pyramid is a free, open GUI tool for offline shader validation and analysis

Pyramid is a free, open GUI tool for offline shader validation and analysis. The UI takes HLSL or GLSL as input, and runs them through various shader compilers and static analyzers.

Dec 20, 2022
Visualization Library is a C++ middleware for high-performance 2D and 3D graphics applications based on OpenGL 1.x-4.x supporting Windows, Linux and Mac OS X.

Visualization Library 2.2 Gallery About Visualization Library is a C++ middleware for high-performance 2D and 3D graphics applications based on the in

Nov 8, 2022
A visualization solution for curriculum and course relationships of Hongkong University of Science and Technology(HKUST)
A visualization solution for curriculum and course relationships of Hongkong University of Science and Technology(HKUST)

Curriculum Map A visualization solution for curriculum and course relationships of Hongkong University of Science and Technology(HKUST) Notice: Qt v6.

Dec 21, 2022
Using D2XX driver to capture UART data to a binary file

What is it This is the sample code to read UART data using any baud rate that FTDI supports and save them to a binary file. How to use Usage: ./larger

Nov 24, 2021
C++ implementation for a binary data storage format.

bsmlib- A C++ library for loading and writing binary data to and from files. bsmlib provides functions for loading, modifying, and saving BSM (Binary

Oct 9, 2022
macOS Endpoint Security Message Analysis Tool

Endpoint Security Message Analysis Tool - esmat esmat is a command line tool for macOS that allows you to explore the behavior of Apple's Endpoint Sec

Sep 27, 2022
Real-time 2D fluid simulator with lots of visualization options.
Real-time 2D fluid simulator with lots of visualization options.

Fluid Simulator Building Start by cloning the program and all submodules using the following command: git clone --recursive https://github.com/linusmo

Dec 14, 2022
Fluid Visualization - The code compilation is only tested on Arch Linux x86_64

Fluid Visualization The code compilation is only tested on Arch Linux x86_64, Linux kernel 5.15.13-arch1, with gcc 11.1.0, CMake 3.22.1, Xorg X server

Jan 30, 2022
Superposition Wave Function Visualization for free particle
Superposition Wave Function Visualization for free particle

Superposition Wave Function Visualization for free particle This is a cross platform software that visualizes the Amplitude(the square root of PDF) an

Nov 27, 2022
Serial Data Monitor is a multiplatform (Windows, Linux, Mac, ...) tool to interactively receive/edit/monitor data and send commands to an embedded system via the serial bus
Serial Data Monitor is a multiplatform (Windows, Linux, Mac, ...) tool to interactively receive/edit/monitor data and send commands to an embedded system via the serial bus

See wiki for full documentation Serial Data Monitor Description Serial Data Monitor is a multiplatform (Windows, Linux, Mac, ...) tool to interactivel

Oct 29, 2021
Sandbox binary and source code for the Siggraph 2017 paper "Water Wave Packets" by Stefan Jeschke (NVIDIA) and Chris Wojtan (IST Austria)

----------------------------- Manual for wave packet viewer ----------------------------- System requirements: Windows8/8.1/10 with DirectX runtime e

Nov 28, 2022
It includes our specialized binary payload DSL (parser and schemas), clients and sdks.

Symbol Monorepo In Q1 2021, we consolidated a number of projects into this repository. It includes our specialized binary payload DSL (parser and sche

Dec 15, 2022
CVE-2021-4034 POC and Docker and Analysis write up
CVE-2021-4034 POC and Docker and Analysis write up

CVE-2021-4034 POC and Docker and Analysis write up

Oct 22, 2022
CVE-2021-3156 POC and Docker and Analysis write up

CVE-2021-3156 [toc] 漏洞简介 漏洞编号: CVE-2021-3156 漏洞产品: sudo 影响版本: 1.8.2-1.8.31sp12; 1.9.0-1.9.5sp1 利用后果: 本地提权 源码获取: https://www.sudo.ws/getting/source/ 环境

Oct 22, 2022
CVE-2022-0185 POC and Docker and Analysis write up
CVE-2022-0185 POC and Docker and Analysis write up

CVE-2022-0185 linux 内核提权(逃逸) [toc] 漏洞简介 漏洞编号: CVE-2022-0185 漏洞评分: 漏洞产品: linux kernel - fsconfig syscall 影响范围: linux kernel 5.1-rc1 ~ 5.16.2 利用条件: linu

Dec 4, 2022