A free, fast, cross-platform volumetric codec for everyone.

Universal Volumetric

The open source Universal Volumetric (".uvol") compressed interchange format for streaming mesh sequences. This project also includes a cross-platform player implementation using h.264 video for texture.

The initial version focuses on performance. Currently, single material mesh sequences of any length are supported. The next version will focus on higher compression ratio, streamability and integration into USD and glTF.

The current implementation uses the MIT-licensed Corto codec from CNR-ISTI Visual Computing Group, which has fast compression and especially fast decompression characteristics.

Collaborators Wanted!

If you are proficient in C++, python, JS, Unity/C#, or you want to support this project creatively or financially, please get in touch!

Example

This tech was built in partnership with Wild Capture and others. You can see an example on Wild Capture's site, here: http://wildcapture.co/volumetric.html

Current uvol files consist of a .uvol binary, manifest file and video texture. Future versions will embed everything into the uvol binary or in a single MP4 container.

Currently playback works in WebGL with three.js and Unity. Android and iOS are in development, Unreal support is on the roadmap (intrepid C++ developers should be able to port this in a day by reading the source from other examples, since the core codec is C++ based).

Requirements

For encoding, you will need Node.js 12+ and Python 3 installed.

For decoding, currently WebGL is supported (especially three.js), Unreal and Unity will come in the next release.

You will need a mesh and texture sequence in OBJ/PNG or OBJ/PLY.

Encoded .uvol files are cross platform, but currently the decoder is written for the web only. Want Unity, Unreal, examples in PlayCanvas and Babylon, etc? Submit and issue and sponsor our project: https://opencollective.com/xrengine

Or you can get in touch with the maintainer directly for any questions about who, what, where and how: [email protected]

Or find us on Discord! https://discord.gg/2amVGBE3a3

Encoding

Mesh

First, you need a .ply sequence. If you have a .obj sequence, you can convert to to ply using this command:

bash src/encoder/make_plys.sh

Next, put your ply files in the src/encoder/encode folder. If you want to change this path, you can do so by modifying Encoder.js (NOTE: this will be moved to an argument in the next version).

Run the encoder on the meshes:

node ./src/Encoder.js example.uvol

If you want to set a specific framerange or frame rate, you can set that as well

// Extended Example: 25 FPS, 500 frames
node ./src/Encoder.js example.uvol 25 0 499

Texture

Texture is stored as an H264 video.

Due to inadequacies in iOS frame sync (as well as multithreaded framesync issues in Unity) we are baking the frame number directly into the texture. This frame sync is 8px high, 128px wide. After some experimentation, we found that this is resistant to aliasing and macroblocking in video. However, it might cause issues with your textures unless you pre-process your textures be offset by 8 px from the bottom of your image. The next version will autoscale your UVs to have 8px available at the bottom.

Encoding frame counter to texture

python3 src/encoder/texture_encoder.py

Encoding image sequence to MP4

ffmpeg -r 30 -s 1024x1024 -i src/encoder/encode/tex%05d.png -vcodec libx264 -crf 25 example.mp4
Owner
XR Foundation
A metaverse for everyone.
XR Foundation
Comments
  • Update Loading effect

    Update Loading effect

    • update logic for the loading effect
    • adding event handler
    • restructure
    • fix some unexpected issue. [issue remained]
    • the html video player is crashed on the low performance
    • doesn't work smoothly on the low internet traffic environment
  • SyntaxError: The requested module 'three' does not provide an export named 'default'

    SyntaxError: The requested module 'three' does not provide an export named 'default'

    "import THREE from 'three';" is error : "default export" in es6 package language

    [email protected]:/opt/UniversalVolumetric-main# node ./src/Encoder.js yifu.uvol
    file:///opt/UniversalVolumetric-main/src/libs/THREECORTOLoader.js:1
    import THREE from 'three';
           ^^^^^
    SyntaxError: The requested module 'three' does not provide an export named 'default'
        at ModuleJob._instantiate (node:internal/modules/esm/module_job:127:21)
        at async ModuleJob.run (node:internal/modules/esm/module_job:191:5)
        at async Promise.all (index 0)
        at async ESMLoader.import (node:internal/modules/esm/loader:331:24)
        at async loadESM (node:internal/process/esm_loader:88:5)
        at async handleMainPromise (node:internal/modules/run_main:65:12)
    
    Node.js v17.1.0
    
  • Playlist feature

    Playlist feature

    • Remove original paths and add new model paths array
    • The video and manifest file names are generated from the model pathname
    • Add play mode (Loop/Single/Single Loop/Random)
  • Update dependency ts-loader to v9.4.2

    Update dependency ts-loader to v9.4.2

    Mend Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | ts-loader | 9.4.1 -> 9.4.2 | age | adoption | passing | confidence |


    Release Notes

    TypeStrong/ts-loader

    v9.4.2

    Compare Source


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update dependency gradle to v7.6

    Update dependency gradle to v7.6

    Mend Renovate

    This PR contains the following updates:

    | Package | Update | Change | |---|---|---| | gradle (source) | minor | 7.5.1 -> 7.6 |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update dependency prettier to v2.8.0

    Update dependency prettier to v2.8.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | prettier (source) | 2.7.1 -> 2.8.0 | age | adoption | passing | confidence |


    Release Notes

    prettier/prettier

    v2.8.0

    Compare Source

    diff

    🔗 Release Notes


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update dependency typescript to v4.9.3

    Update dependency typescript to v4.9.3

    Mend Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | typescript (source) | 4.8.4 -> 4.9.3 | age | adoption | passing | confidence |


    Release Notes

    Microsoft/TypeScript

    v4.9.3: TypeScript 4.9

    Compare Source

    For release notes, check out the release announcement.

    Downloads are available on:

    Changes:

    See More

    This list of changes was auto generated.


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update dependency androidx.test.espresso:espresso-core to v3.5.0

    Update dependency androidx.test.espresso:espresso-core to v3.5.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | androidx.test.espresso:espresso-core | 3.4.0 -> 3.5.0 | age | adoption | passing | confidence |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update dependency androidx.test.ext:junit to v1.1.4

    Update dependency androidx.test.ext:junit to v1.1.4

    Mend Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | androidx.test.ext:junit | 1.1.3 -> 1.1.4 | age | adoption | passing | confidence |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update dependency three to v0.147.0

    Update dependency three to v0.147.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | three (source) | 0.146.0 -> 0.147.0 | age | adoption | passing | confidence | | @types/three (source) | 0.144.0 -> 0.146.0 | age | adoption | passing | confidence |


    Release Notes

    mrdoob/three.js

    v0.147.0

    Compare Source


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about these updates again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update dependency @ianvs/prettier-plugin-sort-imports to v3.7.1

    Update dependency @ianvs/prettier-plugin-sort-imports to v3.7.1

    Mend Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | @ianvs/prettier-plugin-sort-imports | 3.4.2 -> 3.7.1 | age | adoption | passing | confidence |


    Release Notes

    ianvs/prettier-plugin-sort-imports

    v3.7.1

    Compare Source

    What's Changed

    Full Changelog: https://github.com/IanVS/prettier-plugin-sort-imports/compare/v3.7.0...v3.7.1

    v3.7.0

    Compare Source

    What's Changed

    Features

    If you would like to order type imports differently from value imports, you can use the special <TYPES> string. This example will place third party types at the top, followed by local types, then third party value imports, and lastly local value imports:

    "importOrder": ["<TYPES>", "<TYPES>^[./]", "<THIRD_PARTY_MODULES>", "^[./]"],
    
    Dependencies

    Full Changelog: https://github.com/IanVS/prettier-plugin-sort-imports/compare/v3.6.0...v3.7.0

    v3.6.0

    Compare Source

    What's Changed

    Features:

    Add this to your prettier config file to get autocompletion and type checking:

    // @&#8203;ts-check
    
    /** @&#8203;type {import(https://github.com/ianvs/prettier-plugin-sort-imports).PrettierConfig} */
    module.exports = { // ... your config
    
    Maintenance:

    Full Changelog: https://github.com/IanVS/prettier-plugin-sort-imports/compare/v3.5.2...v3.6.0

    v3.5.2

    Compare Source

    What's Changed

    New Contributors

    Full Changelog: https://github.com/IanVS/prettier-plugin-sort-imports/compare/v3.5.1...v3.5.2

    v3.5.1

    Compare Source

    What's Changed

    Full Changelog: https://github.com/IanVS/prettier-plugin-sort-imports/compare/v3.5.0...v3.5.1

    v3.5.0

    Compare Source

    What's Changed

    New Contributors

    Full Changelog: https://github.com/IanVS/prettier-plugin-sort-imports/compare/v3.4.2...v3.5.0


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update dependency @babel/core to v7.20.5

    Update dependency @babel/core to v7.20.5

    Mend Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | @babel/core (source) | 7.19.6 -> 7.20.5 | age | adoption | passing | confidence |


    Release Notes

    babel/babel

    v7.20.5

    Compare Source

    :eyeglasses: Spec Compliance
    • babel-helpers, babel-plugin-transform-destructuring, babel-plugin-transform-modules-commonjs, babel-preset-env, babel-runtime-corejs2, babel-runtime-corejs3, babel-runtime, babel-traverse
    • babel-cli, babel-helpers, babel-plugin-proposal-class-properties, babel-plugin-proposal-class-static-block, babel-plugin-transform-classes, babel-plugin-transform-runtime, babel-preset-env
    • babel-helper-create-class-features-plugin, babel-helpers, babel-plugin-proposal-decorators, babel-plugin-proposal-private-property-in-object, babel-preset-env, babel-runtime-corejs2, babel-runtime-corejs3, babel-runtime
    :bug: Bug Fix
    • babel-parser
    • babel-helper-wrap-function, babel-preset-env, babel-traverse
    • babel-plugin-transform-arrow-functions, babel-plugin-transform-parameters, babel-traverse
    • babel-helpers, babel-node, babel-plugin-proposal-async-generator-functions, babel-plugin-transform-regenerator, babel-preset-env, babel-runtime-corejs2, babel-runtime-corejs3, babel-runtime
    • babel-helper-create-regexp-features-plugin
    • babel-parser, babel-types
    • babel-generator
    • babel-plugin-transform-block-scoping, babel-traverse
    :nail_care: Polish
    :house: Internal

    v7.20.2

    Compare Source

    :bug: Bug Fix
    • babel-core, babel-helper-create-class-features-plugin, babel-helper-module-transforms, babel-helper-plugin-utils, babel-helper-simple-access, babel-node, babel-plugin-transform-block-scoping, babel-plugin-transform-classes, babel-plugin-transform-react-constant-elements, babel-preset-env, babel-standalone, babel-types
    • babel-plugin-transform-typescript
    • babel-parser
    • babel-generator
    • babel-plugin-proposal-decorators, babel-plugin-proposal-object-rest-spread, babel-plugin-transform-jscript
    • babel-plugin-transform-destructuring

    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Suggestion: 4-byte alignment for uvol meshes

    Suggestion: 4-byte alignment for uvol meshes

    A suggestion:

    It looks like the Corto library requires 4-byte alignment for buffers being decoded, but since uvol files do not enforce any alignment, the current Unity code is forcing a new buffer allocation and copy for each frame mesh.

    It seems that both the allocation and the copy could be avoided if the uvol files created enforced 4-byte alignment for each frame mesh...

  • Dependency Dashboard

    Dependency Dashboard

    This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

    Open

    These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

    Detected dependencies

    gradle
    android/gradle.properties
    android/settings.gradle
    android/build.gradle
    • com.android.tools.build:gradle 7.3.1
    android/app/build.gradle
    • androidx.appcompat:appcompat 1.5.1
    • com.google.android.material:material 1.7.0
    • androidx.constraintlayout:constraintlayout 2.1.4
    • com.jakewharton.timber:timber 5.0.1
    • junit:junit 4.+
    • androidx.test.ext:junit 1.1.4
    • androidx.test.espresso:espresso-core 3.5.0
    gradle-wrapper
    android/gradle/wrapper/gradle-wrapper.properties
    • gradle 7.6
    npm
    encoder/dev/package.json
    encoder/package.json
    • esm 3.2.25
    • http-range-fetcher 1.5.0
    • three 0.147.0
    • three-obj-loader-cjs-module 1.0.0
    • xmlhttprequest 1.8.0
    • @babel/cli 7.19.3
    • @babel/core 7.19.6
    • @babel/plugin-transform-modules-commonjs 7.19.6
    • @types/glob 8.0.0
    • ts-loader 9.4.2
    • ts-node 10.9.1
    • typescript 4.9.3
    • yarn >= 0.18.0
    package.json
    • shelljs 0.8.5
    • vite ^3.0.0
    • xmlhttprequest 1.8.0
    • @babel/cli 7.19.3
    • @babel/core 7.19.6
    • @types/three 0.146.0
    • @ianvs/prettier-plugin-sort-imports 3.7.1
    • prettier 2.8.0
    • typescript 4.9.3
    • three >0.130.0
    nuget
    unity/Assembly-CSharp.csproj

    • [ ] Check this box to trigger a request for Renovate to run again on this repository
  • three.js -- Set Draw Range

    three.js -- Set Draw Range

    We can/should set a max buffer size and manipulate the buffer with draw ranges, so that we only render the tris we need, but we don't need to create new mesh objects ever.

  • Clean Mesh Attributes

    Clean Mesh Attributes

    The input PLY meshes must have clean data : only Position, UVs, Normals or Velocity.
    Some input OBJ meshes have references to external textures or excess attributes, this can cause texture corruption or sync issues.

Open h.265 video codec implementation.
Open h.265 video codec implementation.

libde265 - open h.265 codec implementation libde265 is an open source implementation of the h.265 video codec. It is written from scratch and has a pl

Nov 30, 2022
Open Source H.264 Codec

OpenH264 OpenH264 is a codec library which supports H.264 encoding and decoding. It is suitable for use in real time applications such as WebRTC. See

Dec 2, 2022
ffmpeg supporting EVC codec and file formats.

ffevc ffmpeg supporting EVC codec and file formats. MPEG-5 Essential Video Coding (EVC) integration with FFmpeg project. It is supported under Linux a

Nov 23, 2022
Olive is a free non-linear video editor for Windows, macOS, and Linux.
Olive is a free non-linear video editor for Windows, macOS, and Linux.

Olive is a free non-linear video editor for Windows, macOS, and Linux.

Nov 30, 2022
🎥 mpv is a free (as in freedom) media player for the command line.
🎥 mpv is a free (as in freedom) media player for the command line.

mpv is a free (as in freedom) media player for the command line. It supports a wide variety of media file formats, audio and video codecs, and subtitle types.

Nov 25, 2022
Even newer Custom Firmware for Xiaomi Cameras based on Hi3518ev200 Chipset. It includes free RTSP, ONVIF and other improvements based on the work by roleoroleo
Even newer Custom Firmware for Xiaomi Cameras based on Hi3518ev200 Chipset. It includes free RTSP, ONVIF and other improvements based on the work by roleoroleo

Why this Yi-Hack-V5 firmware? The answer is simple: missing updates, RTSP and not based on the latest stock firmware (which features improvements and

Dec 2, 2022
Free plugin-based video converter
Free plugin-based video converter

SEZE II Смотреть на русском SEZE is a plugin-based video converter for Windows/Linux. You can write your video filter in C++ (see section "How to make

Nov 21, 2022
Kodi is an award-winning free and open source software media player and entertainment hub for digital media
Kodi is an award-winning free and open source software media player and entertainment hub for digital media

website • docs • community • add-ons Welcome to Kodi Home Theater Software! Kodi is an award-winning free and open source software media player and en

Dec 1, 2022
Free and open-source media player written in C++

Liquid Media Player Free and open-source media player written in C++. Currently in development. Build Guide Windows Install the MSYS2 Building Platfor

Sep 20, 2022
OpenShot Video Library (libopenshot) is a free, open-source C++ library dedicated to delivering high quality video editing, animation, and playback solutions to the world

OpenShot Video Library (libopenshot) is a free, open-source C++ library dedicated to delivering high quality video editing, animation, and playback solutions to the world

Dec 3, 2022
Custom FFMpeg build script which support cross-compile to macOS/iOS/android

FFmpegCompileScript Custom FFMpeg build script which support cross-compile to macOS/iOS/Android Usage: First, init all submodule: git submodule update

Apr 20, 2022
🤟Super fast H.264/H.265 FLV player
🤟Super fast H.264/H.265 FLV player

??Super fast H.264/H.265 FLV player

Nov 26, 2022
simple mp4 player based on rockchip rv1109 platform

mp4player RV1109平台上实现一个简单的 mp4 播放器,主要是本人使用的开发板QT无法播放mp4,应该是没有编译qst所致,因而想利用rockchip平台自有的 功能实现一个简单的播放器。 base目录包含一些基础框架实现,包含信号,线程,时间等,线程和消息泵的实现非常非常简单,因而不

Jul 17, 2022
Cross-platform silk codec wrap library depends on ploverlake/silk.

libSilkCodec Cross-platform silk codec wrap library depends on ploverlake/silk. Clone & Build Linux/Unix like # clone $ git clone https://github.c

Sep 9, 2022
Fast Volume Rendering with Spatiotemporal Reservoir Resampling (Volumetric ReSTIR)
Fast Volume Rendering with Spatiotemporal Reservoir Resampling (Volumetric ReSTIR)

Fast Volume Rendering with Spatiotemporal Reservoir Resampling (Volumetric ReSTIR) Introduction This repo includes the source code for the SIGGRAPH As

Nov 23, 2022
A tiny cross-platform webview library for C/C++/Golang to build modern cross-platform GUIs.

webview for golang and c/c++ A tiny cross-platform webview library for C/C++/Golang to build modern cross-platform GUIs. The goal of the project is to

Nov 1, 2022
OpenCorr is an open source C++ library for development of 2D, 3D/stereo, and volumetric digital image correlation
OpenCorr is an open source C++ library for development of 2D, 3D/stereo, and volumetric digital image correlation

OpenCorr OpenCorr is an open source C++ library for development of 2D, 3D/stereo, and volumetric digital image correlation. It aims to provide a devel

Nov 30, 2022
Offical repo for "Moynihan, M., Ruano, S., Pagés, R. and Smolic, A., 2021. Autonomous Tracking For Volumetric Video Sequences"
Offical repo for

MeshTracker A segmentation-based tracking algorithm for registering volumetric video meshes (ply/obj) in C++. This is the official implementation of t

Nov 7, 2022
Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimensional grid
Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimensional grid

Treexy is a library that implements a compact hierarchical data structure that can store and manipulate volumetric data, discretized on a three-dimens

Nov 21, 2022
Brute force volumetric path tracer written in C++
Brute force volumetric path tracer written in C++

volpt Brute force volumetric path tracer written in C++. WIP. Features Homogeneous medium Hero wavelength sampling for chromatic absorption/scattering

Oct 21, 2022