Update: This is an ongoing project. A fully processed minimal RGB frames dataset for NeRF training is coming soon.
Resources
Preprocessing (as of 2023, using Python 3.10)
- Download the EPIC Kitchens dataset RGB frames and the Epic Fields JSON Data.
- Copy and Paste this code to the Terminal in your preferred directory and run:
git clone https://github.com/1ssb/epic-filters && cd epic-filters && shopt -s extglob && rm -rf !(filters). - Configure the unzipper code with the target directory and run it as
python unzipper.py - Install Libraries:
pip install Ultralytics cv2 opencv-python Pillow tqdm - Run the overlap filter first, followed by the dark filter, then the hands filter, and finally the fit filter (configure before running and check if necessary) using
python {filter_name}.py
Note: It is recommended to use a CUDA-enabled GPU (Tested on an RTX 4090) for the overlap filter. If you prefer to use the CPU, you can modify the code. It is advisable to use concurrency-based Lock methods for transformations in the Overlap filter.
Usage
Hands Filter
Motivation: This filter focuses on selecting images with human hands while minimizing occlusion of the environment. It employs the ultralytics YOLOv8 object detection model to detect persons in images and calculate the bounding box area ratio. Images with ratios below a certain threshold are copied to a destination folder.
Description: The filter consists of two main functions: "person area ratio" and "main." The former calculates the bounding box area ratio of detected persons, while the latter applies this calculation to filter images. The code can be adapted for other filtering criteria or detection models.
Frustrum Overlap Filter
Design Choice: This filter selects frames based on camera frustum overlap. It employs an overlap threshold and target ratio to determine which frames to retain. The code provides flexibility by allowing different overlap reduction methods. Camera pose metadata in quaternion format from the Epic Fields dataset is utilized.
Description: The filter comprises two functions: "calculate frustum overlap" and "select frames." The former computes overlap ratios between frustums using projection methods, while the latter reads JSON data to extract camera and image information. Frames with high overlap are removed until reaching the target frame count.
Post Processing
Dark Filter: This code removes dark images from a specified directory based on a given threshold value.
Fit: This code resizes and center-crops images in a specified directory using the PIL library.
For more details, please refer to the Filter Code Documentation.
Cite as:
@misc{epic-filters, title={Epic Kitchen Filters}, author={Subhransu S. Bhattacharjee}, year={2023}, howpublished={https://github.com/1ssb/epic-filters/}}