A WFH utility to visually indicate user engagement of audio and video

DIY: In meeting indicator - WFH Utility

The need for in meeting indicator at home assets/in-meeting-indicator-meme1.png

So many of you have gotten accustomed to work from home by now. This pandemic has thrown a lot of challenges at our face when most of us are not ready for. While Work From Home is kind of boon, living with it for a long time comes with its own unique challenges. One of the simplest challenge I have faced in the initial days was to make my family understand that I am in a meeting where Mic/Camera/both are activated. It was becoming difficult sometimes to respond back to the family members for very trivial things while I am listening to a crucial conversation on zoom or trying to answer an important question on an ongoing video chat. Of course they understand that I am in work and it is difficult for me to respond. But these things can slowly be the trends at your home.

Gestures are great way to convey the same. But they don't always go smooth on the receiving side especially when you do them for visitors.

As a technologist, we should be able to tackle this better.

  • How about sending text messages to family members when you are on/off meeting(May be overwhelming)
  • Share your calendar with family members? (How to deal with unscheduled ones?)
  • Mobile app to show your availability(Too much of work?)
  • How about placing visual indicators at our desk? (Can be a Good starter)

I have picked up the last idea out of all the above ones which seems doable with relatively little effort. Back to hardware mode.

Outcome:

Goal of in meeting indicator assets/in-meeting-indicator-visual-output.png

Two simple LED light indicators of distinct colours, one for Microphone and another for Camera. Indicators turn on automatically based on your activity on your mac(picked up Mac as a starter) be it on Zoom, Meet, Discord or Facetime etc.

What do I need?

  1. Majority of people use either one or two meeting applications on a daily basis. Check for developer documentation and find out whether they expose any apis to read Mic and Camera usage status.

    Unfortunately, I will not able to do this at least for zoom meetings when I last checked in their developer docs. Thats too disappointing to find out.

  2. An app to detect Mic or Camera usage on your computer.

    Doing this is not so easy especially to work with OS APIs. As I use Mac and I am no expert in OSX development. I am looking for some open source alternatives and found out a tool called "Oversight" from a security researcher Patrick Wardle. More about the tool here

    Again unfortunate that I don't get any apis to query device status. But hold on, the tool writes the device status to a log file which I can scrape. Hoo yeah.

  3. Two 10mm LED bulbs of different colours and jumper wire connectors which I can get from an online store.

  4. I need a controller which has wifi support. I chose NodeMCU V2 for this.

  5. I have a Raspberry Pi with on OLED display running in my living room all the time. I wanted to make this a central hub for any device communications within my home rather than my work laptop.

    Idea was that my work laptop just updates device status to raspberry pi and NodeMCU queries it every few seconds and control the LED state accordingly. Later I also wanted to leverage this for my windows laptop, Mobile and tablet as well.

Interaction diagram of nodeMCU, Raspberry Pi4 and Mac Laptop assets/in-meeting-indicator-overview.png

Here is a short demo video

In Meeting Indicator Demo

On my Raspberry PI

To make things easy, I have assigned static IP to my Raspberry pi4 device based on its Mac address in my home router configuration.

Written a simple Golang HTTP webserver with endpoints to query as well as update device status.

package main

import(
	"net/http"
	"fmt"
	"nodemculistener/devicestate"
)

var devState *devicestate.Indicator

func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("/mic", handleMicState)
	mux.HandleFunc("/camera", handleCameraState)
	mux.HandleFunc("/*", handleUnknown)

	setup()
	if err := http.ListenAndServe(":3000", mux); err != nil {
		fmt.Println("listen error: ", err.Error())
	}
}

func setup() {
	devState = devicestate.NewIndicator()
}

func handleMicState(rw http.ResponseWriter, r *http.Request) {
	if r.Method == http.MethodGet {
		fmt.Println("mic status", devState.IsMicEnabled())
		rw.Write([]byte(fmt.Sprintf("%v", devState.IsMicEnabled())))
	} else if r.Method == http.MethodPut {
		qParams := r.URL.Query()
		devState.SetMic(qParams.Get("state") == "on")
		rw.WriteHeader(http.StatusOK)
	} else {
		rw.WriteHeader(http.StatusBadRequest)
	}
}

func handleCameraState(rw http.ResponseWriter, r *http.Request) {
	if r.Method == http.MethodGet {
		fmt.Println("camera status", devState.IsMicEnabled())
		rw.Write([]byte(fmt.Sprintf("%v", devState.IsCameraEnabled())))
	} else if r.Method == http.MethodPut {
		qParams := r.URL.Query()
		devState.SetCamera(qParams.Get("state") == "on")
		rw.WriteHeader(http.StatusOK)
	} else {
		rw.WriteHeader(http.StatusBadRequest)
	}
}

func handleUnknown(rw http.ResponseWriter, r *http.Request) {
	fmt.Println("unknown req", r)
}

On My Mac

Installed OverSight which starts writing log to a standard location. /System/Volumes/Data/Users//Library/Application Support/Objective-See/OverSight/OverSight.log

There is a catch here:

Oversight has an additional functionality which sends system tray notifications when device is activated or deactivated with information of the process like PID and name etc. The catch lies here, if you click on Allow on the notification (let's say for zoom process) badge, The process will be whitelisted by oversight and you will not further receive notifications and neither will write to log file for Zoom again.

To solve this, I have turned off notifications from the Mac system settings for Oversight application and also cleared all the whitelisted entries from Oversight application.

Script to scrape OverSight logs and updates device status tp Web server running on Raspberry pi

#!/bin/bash

echo "Base URL configured: $BASE_URL"

function updateMicStatus() {
	if [ $1 -eq 0 ]; then
		curl -X PUT "$BASE_URL/mic?state=on"
		echo "mic status updated to on"
	else
		curl -X PUT "$BASE_URL/mic?state=off"
		echo "mic status updated to off"
	fi
}

function updateCameraStatus() {
	if [ $1 -eq 0 ]; then
		curl -X PUT "$BASE_URL/camera?state=on"
		echo "camera status updated to on"
	else
		curl -X PUT "$BASE_URL/camera?state=off"
		echo "camera status updated to off"
	fi
}

cat "/System/Volumes/Data/Users/krishnak/Library/Application Support/Objective-See/OverSight/OverSight.log" | grep "Audio Device" | tail -n 1 | grep -q -w "active"
AUDIO_INIT_STATUS=$?
cat "/System/Volumes/Data/Users/krishnak/Library/Application Support/Objective-See/OverSight/OverSight.log" | grep "Video Device" | tail -n 1 | grep -q -w "active"
CAMERA_INIT_STATUS=$?

updateMicStatus $AUDIO_INIT_STATUS
updateCameraStatus $CAMERA_INIT_STATUS

tail -1f "/System/Volumes/Data/Users/krishnak/Library/Application Support/Objective-See/OverSight/OverSight.log" | awk '/Audio Device/ && /[^[:alpha:]]active/ { system("curl -X PUT $BASE_URL/mic?state=on") } /Audio Device/ && /inactive/ { system("curl -X PUT $BASE_URL/mic?state=off") }' &
tail -1f "/System/Volumes/Data/Users/krishnak/Library/Application Support/Objective-See/OverSight/OverSight.log" | awk '/Video Device/ && /[^[:alpha:]]active/ { system("curl -X PUT $BASE_URL/camera?state=on") } /Video Device/ && /inactive/ { system("curl -X PUT $BASE_URL/camera?state=off") }'

Run the script from Mac automator as an application:

BASE_URL="http://192.168.11.111:3111" /monitor.sh

Creating a automator application on Mac:

Followed the process from here https://michal.karzynski.pl/blog/2013/01/13/how-turn-shell-commands-mac-os-x-services/

assets/create-new-atomator-app.png

assets/run-shell-script-in-automator-app.png

On My NodeMCU

NodeMCU program to listen to device state changes:

#include <Arduino.h>
#include <WiFiManager.h>
#include "RestClient.h"
#include <ESP8266HTTPClient.h>
#include "Config.h" //Keep your URL and SSID credentials in this

#define MICLED D5
#define CAMERALED D6

WiFiManager wifiManager;
String hostname;

int readMicStatus();
int readCameraStatus();
String getServerHostname();

void setup() {
  pinMode(MICLED, OUTPUT);
  pinMode(CAMERALED, OUTPUT);

  WiFi.mode(WIFI_STA); // explicitly set mode, esp defaults to STA+AP
  Serial.begin(9600);
  //reset settings - wipe credentials for testing
  //wm.resetSettings();

  // Automatically connect using saved credentials,
  // if connection fails, it starts an access point with the specified name ( "AutoConnectAP"),
  // if empty will auto generate SSID, if password is blank it will be anonymous AP (wm.autoConnect())
  // then goes into a blocking loop awaiting configuration and will return success result
	// wifiManager.setConfigPortalBlocking(false);

  if(!wifiManager.autoConnect(HOTSPOT_SSID, HOTSPOT_PWD)) {
        Serial.println("Failed to connect");
        delay(5000);
        ESP.restart();
  } else {
        //if you get here you have connected to the WiFi    
        Serial.println("connected...yeey :)");
  }
}

void loop() {
	// wifiManager.process();
  if (readMicStatus() == 1) {
    digitalWrite(MICLED, HIGH);
  } else {
    digitalWrite(MICLED, LOW);
  }

  if (readCameraStatus() == 1) {
    digitalWrite(CAMERALED, HIGH);
  } else {
    digitalWrite(CAMERALED, LOW);
  }

  delay(2000);
}

int readMicStatus() {
  int ret = -1;
  HTTPClient http;
  http.begin(MIC_ENDPOINT);
  int httpCode = http.GET();

  if (httpCode > 0) {
    String payload = http.getString();
    Serial.print("mic status: ");
    Serial.println(payload);
    if (payload == "true") {
      ret = 1;
    } else {
      ret = 0;
    }
  }
  http.end();
  return ret;
}

int readCameraStatus() {
  int ret = -1;
  HTTPClient http;
 
  http.begin(CAMERA_ENDPOINT);
  int httpCode = http.GET();

  if (httpCode > 0) {
    String payload = http.getString();
    Serial.print("camera status: ");
    Serial.println(payload);
    if (payload == "true") {
      ret = 1;
    } else {
      ret = 0;
    }
  }
  http.end();
  return ret;
}

LED wiring:

  • Connect the two LED anodes(longer leg) to D5, D6 pins and grounds(shorter leg) to corresponding ground pins marked as GND of NodeMCU and that's it.

WiFiManager library:

When NodeMCU is not able to connect to your home router/hotspot for various reasons like SSID/password change. Instead of writing new SSID and password configuration by physically connecting and flashing NodeMCU with your computer, it turns on a Wifi Hotspot with pre defined SSID and allows you to connect via mobile/laptop and allows your add/correct router details. This library solves the most common problem of maintenance and I liked it.

Config.h:

Keep your URL and SSID credentials in a separate header file rather than in the code directly and don't forget to ignore it from git tracking. I placed these details in include/config.h file and included it in the main CPP file. This is one of the preferred and simplest approaches for maintaining secrets with CPP.

**PlatformIO.org IDE for IOT development:**

Arduino is a great IDE for programming NodeMCU boards. But it definitely lacks some things which a regular C++ IDE would have like

  • Auto code completion
  • Static code analysis
  • Library manager
  • Debugger
  • Test framework

I personally found PlatformIO IDE solves these problems as well as supporting other existing features like inbuilt board management, serial plotter etc too.

More Ideas/Coming Soon:

  • Trim Oversight code of additional functionality and do better integration
  • Make an API call using Alexa skills/IFTTT to change colour of your smart bulb instead of LEDs
  • Instead of using (Mac + NodeMCU + Raspberry Pi), this setup can be simplified by directly using either of wifi enabled NodeMCU or Raspberry Pi with your Mac as they both support GPIO pins for connecting LEDs.
  • Send push notifications to your family members phones using services like Twilio/Pushbullet etc.

Note about VSCode extensions:

While I generally develop on mac but the actual code is sitting on my Raspberry pi SSD. I use a cool "Remote SSH" VSCode extension to access files over SSH connection on VSCode editor as if they were on your local machine.

Moreover, VSCode is intelligent enough to let me choose my PlatformIDE extension to be installed on the Raspberry pi over SSH thus allowing Platform IDE to detect and work with connected devices on Raspberry pi natively. How cool is that.

Reference Links:


Patrick Wardle - OverSight: Exposing Spies on macos

Disclaimer: Installing a third party tool like OverSight comes up with their own set of questions about security, privacy etc. This tool is open source and go see for yourself before you want to use it.

Early Version: In Meeting indicator LEDs taped to the back of my external monitor assets/nse-970011749336351535-1585045123.jpg

Owner
krishna kumar T
Erlang, Golang, Rust, Elixir, Industrial robotics
krishna kumar T
Similar Resources

SortNode is a JS binding for SORT: Simple, online, and real-time tracking of multiple objects in a video sequence.

SortNode is a JS binding for SORT: Simple, online, and real-time tracking of multiple objects in a video sequence.

Aug 2, 2022

ScanNet is an RGB-D video dataset containing 2.5 million views in more than 1500 scans, annotated with 3D camera poses, surface reconstructions, and instance-level semantic segmentations.

ScanNet is an RGB-D video dataset containing 2.5 million views in more than 1500 scans, annotated with 3D camera poses, surface reconstructions, and instance-level semantic segmentations.

Aug 12, 2022

Video game library manager with support for wide range of 3rd party libraries and game emulation support, providing one unified interface for your games.

Video game library manager with support for wide range of 3rd party libraries and game emulation support, providing one unified interface for your games.

An open source video game library manager and launcher with support for 3rd party libraries like Steam, GOG, Origin, Battle.net and Uplay. Includes game emulation support, providing one unified interface for your games.

Aug 17, 2022

🎬 ScreenToGif allows you to record a selected area of your screen, edit and save it as a gif or video.

🎬 ScreenToGif allows you to record a selected area of your screen, edit and save it as a gif or video.

ScreenToGif 🎬 screentogif.com This tool allows you to record a selected area of your screen, live feed from your webcam or live drawings from a sketc

Aug 10, 2022

Open h.265 video codec implementation.

Open h.265 video codec implementation.

libde265 - open h.265 codec implementation libde265 is an open source implementation of the h.265 video codec. It is written from scratch and has a pl

Aug 4, 2022

Video player for 3ds

Video player for 3ds

Video player for 3DS Patch note v1.0.1 Added allow skip frames option v1.0.0 Initial release Summary Video player for 3DS Performance 256x144(144p)@30

Aug 9, 2022

Plugin for VLC that pauses/plays video on mouse click

Pause Click plugin for VLC VLC plugin that allows you to pause/play a video by clicking on the video image. Can be configured to work nicely with doub

Aug 13, 2022

Real-Time Intermediate Flow Estimation for Video Frame Interpolation filter for VapourSynth

Description RIFE filter for VapourSynth, based on rife-ncnn-vulkan. Usage rife.RIFE(clip clip[, int model=0, int gpu_id=auto, int gpu_thread=2, bint t

Aug 15, 2022

Anki-like app for spaced repetition of video clips

Anki-like app for spaced repetition of video clips

ReeePlayer The ReeePlayer application is designed for spaced repetition of fragments (clips) of video and audio files with similar principle as in Ank

May 2, 2022
OpenShot Video Library (libopenshot) is a free, open-source C++ library dedicated to delivering high quality video editing, animation, and playback solutions to the world

OpenShot Video Library (libopenshot) is a free, open-source C++ library dedicated to delivering high quality video editing, animation, and playback solutions to the world

Aug 11, 2022
Vulkan Video Sample Application demonstrating an end-to-end, all-Vulkan, processing of h.264/5 compressed video content.
Vulkan Video Sample Application demonstrating an end-to-end, all-Vulkan, processing of h.264/5 compressed video content.

This project is a Vulkan Video Sample Application demonstrating an end-to-end, all-Vulkan, processing of h.264/5 compressed video content. The application decodes the h.264/5 compressed content using an HW accelerated decoder, the decoded YCbCr frames are processed with Vulkan Graphics and then presented via the Vulkan WSI.

Aug 15, 2022
Minimalist video maker -- simplify your music score video making process!

VisualScores 极简视频制作程序,简化你的乐谱视频制作! 如果需要编译,请解压 lib 文件夹中压缩包。 使用前请参考 manual 文件夹中的用户手册。 请勿修改、移动或删除 resource 文件夹中的任何文件。 VisualScores Minimalist video maker

Jan 11, 2022
yangwebrtc is a self-developed rtc architecture supporting Webrtc/Srt/Rtmp, including a variety of video and audio codecs and processing, etc.
yangwebrtc is a self-developed rtc architecture supporting Webrtc/Srt/Rtmp, including a variety of video and audio codecs and processing, etc.

YangWebrtc Overview yangwebrtc是一个自主研发的支持Webrtc/Srt/Rtmp的rtc架构,包含多种视音频编解码和处理等。 支持视频会议、高清录播直播、直播互动等多种视音频应用。 可用于远程教育、远程医疗、指挥调度、安防监控、影视录播、协同办公、直播互动等多种行业应用

Aug 5, 2022
NymphCast is a audio and video casting system with support for custom applications.
NymphCast is a audio and video casting system with support for custom applications.

NymphCast is a software solution which turns your choice of Linux-capable hardware into an audio and video source for a television or powered speakers. It enables the streaming of audio and video over the network from a wide range of client devices, as well as the streaming of internet media to a NymphCast server, controlled by a client device.

Aug 16, 2022
theora-player is an embeddable theora video player C++ library based on the libtheora sample. It has no audio support at this moment.

theora-player Description theora-player is an embeddable theora video player C++ library based on the libtheora sample. It has no audio support at thi

Jun 18, 2022
Dolby AC-4 Audio & ATSC 3.0/HEVC Video Playback
Dolby AC-4 Audio & ATSC 3.0/HEVC Video Playback

VideoPlayer Qt+ffmpeg+SDL2+Dolby+AC-4+AC4+HEVC+ATSC3.0+4K+video This is a fork of the video player found here: https://github.com/yundiantech/VideoPla

May 30, 2022
Vireo is a lightweight and versatile video processing library written in C++11

Overview Vireo is a lightweight and versatile video processing library that powers our video transcoding service, deep learning recognition systems an

Aug 10, 2022
Olive is a free non-linear video editor for Windows, macOS, and Linux.
Olive is a free non-linear video editor for Windows, macOS, and Linux.

Olive is a free non-linear video editor for Windows, macOS, and Linux.

Aug 8, 2022
SRS is a simple, high efficiency and realtime video server, supports RTMP/WebRTC/HLS/HTTP-FLV/SRT/GB28181.
SRS is a simple, high efficiency and realtime video server, supports RTMP/WebRTC/HLS/HTTP-FLV/SRT/GB28181.

SRS is a simple, high efficiency and realtime video server, supports RTMP/WebRTC/HLS/HTTP-FLV/SRT/GB28181.

Aug 14, 2022