Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added subchunking, but needs more improvements #16

Merged
merged 12 commits into from
Mar 4, 2023

Conversation

krestenlaust
Copy link
Owner

closes #5

@krestenlaust
Copy link
Owner Author

I still need to implement chunk IDs, instead of just adding delays between packages to ensure they are received in the correct order.

@krestenlaust
Copy link
Owner Author

The problem now is when the packets from the next image catch up with the packets from the previous image (or there is packet loss at the previous one) then the chunk IDs collide and it tries to store a chunk with the wrong size in the previous image and then it crashes.

The solutions I see are the following:

  1. When a packet from a new image arrives, drop the old one as it is already out of date anyway, or there could be packet loss.
    The solution doesn't require much, but I'm afraid that frameskipping can be distracting and make the program appear laggy if it happens often.

  2. When a packet from a new image arrives, replace the chunk in the old image with the new chunk so that part of the image is slightly newer.
    The disadvantage of this solution is that it requires me to change the way images are serialized, as the current one basically just takes an image file and splits it up. Because of this, the image is already compressed somewhat. The compression would disappear, and I would have to implement a new form of compression myself, and it would not be as strong as it has to compress at chunk-level instead of image-level.
    Another disadvantage is that the display of the image on the client should be updated much more often, as it should now be updated every time a packet arrives, rather than every time an entire image is assembled.

The advantage is that this method handles packet loss much better and it just updates the image continuously.

  1. When a packet from a new image arrives, it is stored separately at the same time as it waits for packets from the old image, and if the new one is finished first, the old one is dropped.
    The disadvantage is the still the frameskipping, and it kinda ignores packetloss.

(redone to remove unrelated changes)
@krestenlaust
Copy link
Owner Author

Decided on going with solution 2 would be best, instead of simply delaying the issue of packetloss.

@krestenlaust
Copy link
Owner Author

krestenlaust commented Feb 28, 2023

This PR is a pain. I've been experiencing the one visual bug after the other, right now the visual is skewed when the size of the image is uneven. On top of this, I'm also experiencing problems with the removed compression, as the bandwidth required for sending full HD uncompressed even on loopback adapter requires an high amount of bandwidth: 1920 x 1080 x 3 color bytes x 8 (to convert to bits) = ~50 Megabits per frame. For 30 frames per second that's 1.5 Gbps.

I still need to find a good way to calculate the need chunks for sending a frame

@krestenlaust
Copy link
Owner Author

billede

…*just works*

I guess that's it then. The code should be cleansed before merging, but otherwise it's ready to merge.
@krestenlaust
Copy link
Owner Author

Should be working correctly now, just need to redo some of the code and remove some comments and debug messages.

@krestenlaust krestenlaust marked this pull request as ready for review March 4, 2023 21:35
@krestenlaust krestenlaust merged commit 73d3177 into master Mar 4, 2023
@krestenlaust krestenlaust deleted the subchunked-streams branch March 4, 2023 21:37
@krestenlaust krestenlaust added enhancement New feature or request robustness An issue that relates to making the program more sturdy and robust labels Mar 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request robustness An issue that relates to making the program more sturdy and robust
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement 'chunking' of streams
1 participant