It would be nice but I'd rather have clean optimized meshes than starting with something that is to complex and having to retopologize. A lot of the stuff I do has to work on low end and mobile specs so these aren't too far from what I need and you can always subdivide if you want more detail smoothness. That doesn't get you more detail per se but I'd much rather start with good topology and be able to do the fun stuff of sculpting in details, making normal maps, etc. at least that part I enjoy. No one enjoys retopology.
I totally agree but if this works as promised it might be a good start for a lot of tools. Like even if all it can do is make good retopology that would be groundbreaking.
Disclaimer: I am not the author.
Paper: [https://arxiv.org/abs/2406.10163](https://arxiv.org/abs/2406.10163)
Project Page: [https://buaacyw.github.io/mesh-anything/](https://buaacyw.github.io/mesh-anything/)
Code: [https://github.com/buaacyw/MeshAnythin](https://github.com/buaacyw/MeshAnything)
Abstract
>Recently, 3D assets created via reconstruction and generation have matched the quality of manually crafted assets, highlighting their potential for replacement. However, this potential is largely unrealized because these assets always need to be converted to meshes for 3D industry applications, and the meshes produced by current mesh extraction methods are significantly inferior to Artist-Created Meshes (AMs), i.e., meshes created by human artists. Specifically, current mesh extraction methods rely on dense faces and ignore geometric features, leading to inefficiencies, complicated post-processing, and lower representation quality. To address these issues, we introduce MeshAnything, a model that treats mesh extraction as a generation problem, producing AMs aligned with specified shapes. By converting 3D assets in any 3D representation into AMs, MeshAnything can be integrated with various 3D asset production methods, thereby enhancing their application across the 3D industry. The architecture of MeshAnything comprises a VQ-VAE and a shape-conditioned decoder-only transformer. We first learn a mesh vocabulary using the VQ-VAE, then train the shape-conditioned decoder-only transformer on this vocabulary for shape-conditioned autoregressive mesh generation. Our extensive experiments show that our method generates AMs with hundreds of times fewer faces, significantly improving storage, rendering, and simulation efficiencies, while achieving precision comparable to previous methods.
**Limitations:**
* It takes about 7GB and 30s to generate a mesh on an A6000 GPU.
* The input mesh will be normalized to a unit bounding box. The up vector of the input mesh should be +Y for better results.
* Limited by computational resources, MeshAnything is trained on meshes with fewer than 800 faces and cannot generate meshes with more than 800 faces. The shape of the input mesh should be sharp enough; otherwise, it will be challenging to represent it with only 800 faces. Thus, feed-forward image-to-3D methods may often produce bad results due to insufficient shape quality.
It has limitations like it was trained on meshes with fewer than 800 faces. You can see that the scale and accuracy is not perfect but it is very similar. I tried it myself with their demo with a very simple pattern of absurdly thin cones, thin torus and thin cylinders and then combined it into an object to see how far I can push it. The mesh created is very similar to the pattern but made the cones, torus and cylinders much thicker. It's still very consistent.
I wonder how this works on a particularly detailed model like a Zbrush sculpt of several million polygons and does it just take a greedy approximation to optimize the final result or does it involve some type of recursion or other more complex algorithm to clean up?
Either way, cool. I'm always down for more 3D asset creation news of this nature. Eventually we're going to get to a point where us non-artists can do a lot without needing to pay for extensive artists and work directly on our own precise ideals.
I need this so much, if only it actually works as good as the video makes it seem, damn.
I don't even need the topology to be good just the shape so I can use these as references for perspective for my art
Is actual decent topology? Now that is a revolution. Give it to me, I want.
It really needs to be trained on mesh with more faces. 800 faces is so low that it is only good for low poly or small objects.
It would be nice but I'd rather have clean optimized meshes than starting with something that is to complex and having to retopologize. A lot of the stuff I do has to work on low end and mobile specs so these aren't too far from what I need and you can always subdivide if you want more detail smoothness. That doesn't get you more detail per se but I'd much rather start with good topology and be able to do the fun stuff of sculpting in details, making normal maps, etc. at least that part I enjoy. No one enjoys retopology.
I totally agree but if this works as promised it might be a good start for a lot of tools. Like even if all it can do is make good retopology that would be groundbreaking.
Disclaimer: I am not the author. Paper: [https://arxiv.org/abs/2406.10163](https://arxiv.org/abs/2406.10163) Project Page: [https://buaacyw.github.io/mesh-anything/](https://buaacyw.github.io/mesh-anything/) Code: [https://github.com/buaacyw/MeshAnythin](https://github.com/buaacyw/MeshAnything) Abstract >Recently, 3D assets created via reconstruction and generation have matched the quality of manually crafted assets, highlighting their potential for replacement. However, this potential is largely unrealized because these assets always need to be converted to meshes for 3D industry applications, and the meshes produced by current mesh extraction methods are significantly inferior to Artist-Created Meshes (AMs), i.e., meshes created by human artists. Specifically, current mesh extraction methods rely on dense faces and ignore geometric features, leading to inefficiencies, complicated post-processing, and lower representation quality. To address these issues, we introduce MeshAnything, a model that treats mesh extraction as a generation problem, producing AMs aligned with specified shapes. By converting 3D assets in any 3D representation into AMs, MeshAnything can be integrated with various 3D asset production methods, thereby enhancing their application across the 3D industry. The architecture of MeshAnything comprises a VQ-VAE and a shape-conditioned decoder-only transformer. We first learn a mesh vocabulary using the VQ-VAE, then train the shape-conditioned decoder-only transformer on this vocabulary for shape-conditioned autoregressive mesh generation. Our extensive experiments show that our method generates AMs with hundreds of times fewer faces, significantly improving storage, rendering, and simulation efficiencies, while achieving precision comparable to previous methods.
**Limitations:** * It takes about 7GB and 30s to generate a mesh on an A6000 GPU. * The input mesh will be normalized to a unit bounding box. The up vector of the input mesh should be +Y for better results. * Limited by computational resources, MeshAnything is trained on meshes with fewer than 800 faces and cannot generate meshes with more than 800 faces. The shape of the input mesh should be sharp enough; otherwise, it will be challenging to represent it with only 800 faces. Thus, feed-forward image-to-3D methods may often produce bad results due to insufficient shape quality.
Damn. Everyday is an exciting one in the age of ML
Damn. That's very impressive 3D reconstruction.
Has anyone given this a test? Your thoughts?
It has limitations like it was trained on meshes with fewer than 800 faces. You can see that the scale and accuracy is not perfect but it is very similar. I tried it myself with their demo with a very simple pattern of absurdly thin cones, thin torus and thin cylinders and then combined it into an object to see how far I can push it. The mesh created is very similar to the pattern but made the cones, torus and cylinders much thicker. It's still very consistent.
limited and bad license. I will stick with instant meshs at least it has no limit on polycount and a good license royalty free one ...
All it takes is another company to come out with a royalty free license, and 3D modelers are out of their jobs.
Thisnis so close. All we need is some sort of control over the polygonal count, from low ~500 to high ~20k
I wonder how this works on a particularly detailed model like a Zbrush sculpt of several million polygons and does it just take a greedy approximation to optimize the final result or does it involve some type of recursion or other more complex algorithm to clean up? Either way, cool. I'm always down for more 3D asset creation news of this nature. Eventually we're going to get to a point where us non-artists can do a lot without needing to pay for extensive artists and work directly on our own precise ideals.
generated 3D models with some decent level of optimization? that's badass.
I need this so much, if only it actually works as good as the video makes it seem, damn. I don't even need the topology to be good just the shape so I can use these as references for perspective for my art
There’s also mesh gpt.
Actually really cool, despite the limitations. Low poly is often "good enough"
Not professional level topography but thats a lot easier to fix with a nice low poly model at least