Help with dynamic speed up/down

[Implosiv3]

Hi, I need some theoretical help about video dynamic speed up/down. I think this post is targeting the main and old contributors here.

I'm building an automated video editor engine on top of moviepy and I'm trying to implement video speed up/down based on dynamic speed_change factors. I mean, I know the current moviepy system is ready to apply a factor to the whole clip, so if you use a 0.5x factor to a 1s video, it will end up with a duration of 2s because each frame is duplicated, but this goes farther.

In this case, you are able to build a dynamic graphic function that gives you an array of speed_change factors for each frame. See this image below:

This image defines the array. Imagine it as this one: [1, 1.5, 2.7, 1.3, 0.8, 1.0]. This is the speed_change factors array you get from the graphic (not the one in the image, the image is just to clarify).

I've built a system to iterate over all the frames, one by one, and build a new frames array with the frames needed to fit that speed_change factors array. This is simple with frames, but now I'm working with audio that is more difficult.

As I'm doing it manually and trying to understand deeply how t and eveything works in this awesome moviepy engine, do you think there is a better way, using things from moviepy, to achieve this? What comes to my mind is something like: if you need to duplicate the frame, hold the previous t for the next iteration so the frame obtained will be the same. But I think this could break the moviepy system or something (haha).

I've tried to use the time_transform method, but it tells me that there is no duration set and it is not as simple as multiplying the duration by a factor because it is an array of them. As I said, I don't understand how t works deeply at all, but I'm trying to ;), sorry about that.

Thank you very much in advance for any word you spend the time to write me :)

[OsaAjani]

That one is kind of hard, but I think the way to go would be with a custom time transform. I'm not sure exactly how it should work, but the basic principle of time transform is that when rendering a video we ask "give me the frame a time t" and time transform actually kind of intercept and modify that time t. Meaning we will get the frame pointed by this new t.

So I guess that if you have a serie of timestamps and speed, let say ([0, 2, 0.5], [3, 5, 1.5]), where those numbers represent start of period, end of period, speed factor. You could write a time transform function that for any t could see where it falls in that list and apply the proper changes. Not sure but I think you might need to consider previous transformation of all t < target t to get the correct time.

If you manage to do it, make it a real effect and please pr, that seems cool as f.

[Implosiv3]

Thank you. I'm going deeper in the code to finally understand how t works. By now, I've been able to do something close to what I want.

I'm trying to apply this speed factor graphic, which starts on 0.1x, goes to 2x just in the middle, and then back to 0.1x at the end:

I'm using a video of 2 seconds, and this is my result:
https://github.com/user-attachments/assets/339d0331-683b-4564-a3e6-3e39e01918cd

And this is the result, doing the same, with another famous video editor:
https://github.com/user-attachments/assets/f6f54477-6168-40de-8451-adeb81d38ad4

The audio in my video is like glitched, I don't like it, because it is raw, but the one on the video editor result is better (but they are doing some audio treatment I don't). But, according to the image only, I think my solution is better because it fits the graphic more than the video editor. The original video lasts 2 seconds, and my result is 4 seconds long, that makes sense, but the one from the video editor lasts 2 seconds or less, and it is weird according to the graphic.

A 0.1x factor means 1 frame is repeated 10 times, so duration is 10x times longer, and a 2x means you skip one frame, so duration is the half. Thats why 10x times longer vs 2x times shorter makes me think my video is more faithful to the graph.

By the way, currently this is being done using not the t for the audio, so I keep working on it to understand and make it moviepy-effect adaptable (and easier).

What do you think about the effect?