Depth Anything 3 Tests - SUPER IMPRESSIVE RESULTS! #531
Replies: 23 comments 54 replies
-
|
after some more testing it does seem to have some problems with crushing and clipping into pure white values and the falloff seems to be very narrow by default where a lot of the foreground is clipped white and a lot of the background will be clipped black making the resulting 3D look like 2D billboards instead of true 3D, idk if this is an issue with the gui i am using currently or a model issue, i would guess probably a gui issue, are there any plans to see if da3 will work with iw3 in the near future? |
Beta Was this translation helpful? Give feedback.
-
|
I haven't looked at it much yet, |
Beta Was this translation helpful? Give feedback.
-
|
I ran another test on a 30 second movie clip, any scene that includes the sky seems to degrade the image quite a bit overall, at 6 seconds the snow machine fog thing is picked up very well, at 16 seconds as you can see why the sky isnt bugging the whole shot its very crisp and detailed, even picks up individual hair strands, its still overblown though into white so you cant see any detail on the actual face, at 20 seconds the dust flying past is picked up really well and the particles of sand that pop up look like they all get picked up too even some of the water at 27 seconds is picked up, although not the main rain which i think is a bit unrealistic anyway. Zootopia_Depth.mp4Reference: Zootopia_Ref.mp4With all the bugs sorted out this will be a hell of a lot more powerful and quicker than any of the current models used in iw3, cant wait will its fully working |
Beta Was this translation helpful? Give feedback.
-
|
the author of Depth-Anything-3-GUI implemented the patch and it has drastically improved the quality, here is a before and after, both runs were at 1080p res: Before: Snowy.Forest.30Fps.Depth.2.mp4After: Snowy.Forest.30Fps.Depth.1.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
ok this is pretty insane, check this out, this was run at 3402x1428 with the new fix, i dont think ive ever seen a depth map so clear in my life, straight up looks like a z-depth buffer of a 3D scene: Zootopia.2016.4K.Hdr.Dv.2160P.Bdremux.Ita.Eng.X265-Nahom.Depth.mp4just a short snippet for now, full 30 seconds is processing |
Beta Was this translation helpful? Give feedback.
-
|
Here is the full clip, the fix did work overall a lot better but there is still some obvious artifacting around some edges and on smoke and particles, the smoke and particles in particular look worse than before imo Zootopia.2016.4K.Hdr.Dv.2160P.Bdremux.Ita.Eng.X265-Nahom.Depth.1.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
I also added Any_V3_Mono to iw3. |
Beta Was this translation helpful? Give feedback.
-
|
used 1080p depthmap to get this video, used the inpainting too, its cross eye view mode, i think its very good at a glance, would have to watch in vr to make sure though: Snowy.Forest.30Fps.Rlf.Cross.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
it seems to be working great all things considered, looks amazing with the inpainting, only things really are the weird artefacts from the sky sometimes and (maybe) the increased vram usage but idk if its just overhead from other stuff the program is doing or if its not running as optimally as it could be, anyway great job implementing it so fast and even adding it to the inpaint branch |
Beta Was this translation helpful? Give feedback.
-
|
@nagadomi ... as always your dedication and speed in order to implement and thus support the latest ai-models just makes me very grateful. So much appreciated! Currently I am running my first full feature conversion setting depth resolution at 518 VRAM usage with 10,1 GByte is just fine and comparable with VDA_L with the same settings. In my humble opinion nothing special to report here. Unfortunately when it comes especially to those nasty distortions at very narrow structures I had hoped that DAV3 would have been much better than any previous versions. But... no... right now it isn't. But see for yourself... As soon as the depth resolution has been increased, in this example to 1080, the distortion largely disappears and the narrow structure is much better aligned, showing here... ... which of course drops the average fps conversion speed dramatically. Both example images are taken from an interlaced screenshot, meaning, the left and right eye picture are merged together from a Full SBS conversion into one frame for a better comparison. But as @nagadomi already mentioned this is just a first implementation of DAV3 with plenty of space for improvements. Anyways, looking forward how my first entire feature film is going to perform on a big screen tonight using this brand new ai-model. Thanks for each and everything and write to you soon... TeeJay-NLD |
Beta Was this translation helpful? Give feedback.
-
|
just convert a 5min clip with the new depth model but still not as good as vda the new model as a issues with the depth it can't keep the depth it drop the depth in some frames it is not as good as vda which l find no issue with the depth |
Beta Was this translation helpful? Give feedback.
-
|
Ok, I've figured out the vram problem kind of, when using the export format it seems to behave exactly like the gui does, using around the same for the same res, but as soon as anything else is involved it jumps significantly, even when its not doing any stereo stuff like if i select RGB-D it still uses around 5Gb more vram than just export, is this a bug or intended? and if it is, is there no way that you could run the stereo step separately from the depth step? i know there is the export and .yml method but is there no all in 1 solution without manually going back for each video done? this would be especially helpful for things like the inpainting fill as it uses a lot of vram by default limiting depth res if you were to do it all in 1. Here is a test render of the Zootopia scene done in iw3 this time, the things that don't seem to be working right is of course the sky mask having artifacts and it seems some depth values are off as you can see at around 6 seconds with the stadium lights, the ground has no depth at all and is entirely flat, not sure if this is correct or not but looks kind of odd, apart from that and the flickering it seems to be working as intended: iw3_Zootopia_1428x3402.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
Here's something I came across. A drop in the 3D. Maybe something to do with Scene Boundary Detection or Flicker Reduction? These dramatic drops in 3D were happening on and off throughout this 20minute film with DA3. |
Beta Was this translation helpful? Give feedback.
-
|
i tested the same scene on the new patch and some areas have less of the glitches than before but some scenes seem to have worse glitches than before for some reason: iw3_Zootopia_1428x3402_new_patch.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
Hi @nagadomi, I am not quite sure if the following question relates to this topic but please allow me to try. I noticed a new setting "Inpaint Max With" within the IW3 GUI. Unfortunately, I could not find any explanation on this in the given documentation. Therefore, I am curious on how this setting affect the overall inpainting (quality)? Does this setting only relates to certain "Depth Models" or other settings? Sure, I can take a look at the underlying changes to the code but some describing words would be much more helpful at least for me. If I have overlooked any, I sincerely apologize. Regarding your extensive to do-list I am super curious what comes next for IW3 :-). Thanks for reading and as always happy coding... TeeJay-NLD |
Beta Was this translation helpful? Give feedback.
-
|
Some comparisons... I used Any V3 Mono and VDA. I tried Any V2 but the results were very similar to VDA so... Windows and Glass - DA3 misses most of the 3D separation of the glass and windows in these scenes but did better with catching the edge of the sliding glass (girl scene). There's 2 vid clips to compare. Knive Blade |
Beta Was this translation helpful? Give feedback.
-
|
I've done some more testing with da3 and it seems the quality/accuracy of the depth degrades the higher the res goes, around 1080p seems like the sweet spot where its only a tiny bit less stable then the 504 res training resolution, the issues with the background/sky seem to be a lot worse the higher res but still show up on the 504 res version, TTA setting seems to work but i dont notice much or any difference in quality or stability |
Beta Was this translation helpful? Give feedback.
-
|
I tested it with various images, and when there's no depth in front of the subject, it looks very good. |
Beta Was this translation helpful? Give feedback.
-
|
Hello, @nagadomi or anyone, can you please help me? Can you tell me how to install Depth Anything V3, since it’s in safetensors format and not pth? When I update IW3, I only see Any_V3_Mono. I’d love to try the others. I don’t know if there’s anywhere that explains updates and their details when they’re released, but if so, it would be great to know, especially for those of us who aren’t computer-savvy and use Windows. Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
@nagadomi I've done a couple more tests, I used 2 clips from the game Arc Raiders that I recorded, one of the clips is fully outside and the other fully inside. Outside clip: Arc_Outside_1080p.mp4Inside clip: Arc_Inside_1080p.mp4As you can see the clip outside has serious degradation and errors compared to the inside clip, idk if it is fixable currently but i have looked at the official Depth Anything 3 github and they seem to have added a lot more stuff which might help fix the problem. |
Beta Was this translation helpful? Give feedback.
-
|
Just tested the new patch and the depth scale seems more stable overall, used the same settings as last time: Zootopia_3402x1428_Depth.mp4at the 3 second mark when the text shows up at the bottom the old version used to shift the entire scenes depth a lot darker, seems to stay the same now, overall there might be a tiny bit less flicker too but its hard to tell, unfortunate that the model has a problem with the sky thing, i hope they patch it sometime or at least hope it wont be and issue if they release a video version like VDA |
Beta Was this translation helpful? Give feedback.
-
|
I ran a test after 'Update' with those clips I shared from above. There's less of a depth jump between some scenes... but also there's a little less depth in general for the same depth settings. It still is not showing 3D behind glass windows. |
Beta Was this translation helpful? Give feedback.
-
|
Even with strength at 5, Any_V3_Mono and Any_V3_Mono_01 produce a flat subject (or almost flat) when the subject is very close to the camera. |
Beta Was this translation helpful? Give feedback.




















Uh oh!
There was an error while loading. Please reload this page.
-
ive tested da3 and it is a generational upgrade over 2, i can run full 1080p videos at 1920x1080 depth resolution that runs faster and looks better than VDA_L and other things like depthcrafter, the only issues i have run into so far is slight flicker and outside scenes with the sky seem kind of bugged, as well as things with a solid colour background, i think it uses a new feature for those which was not working properly in the gui i used for these tests, i used this gui here:
https://github.com/enoky/Depth-Anything-3-GUI
here are some video examples of the depth maps:
001.Depth.mp4
Img.0997.Depth.1.mp4
Img.1183.Depth.1.mp4
Beta Was this translation helpful? Give feedback.
All reactions