Skip to content

Depth Estimation Agent

DepthEstimationAgent

Bases: SensoryAgent

A depth estimation agent that uses a remote depth estimation server to estimate depth from an image.

Examples:

agent = DepthEstimationAgent(model_src="https://api.mbodi.ai/sense/") result = agent.act(image=Image("resources/xarm.jpeg", size=(224, 224)))

Source code in mbodied/agents/sense/depth_estimation_agent.py 
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
class DepthEstimationAgent(SensoryAgent):
    """A depth estimation agent that uses a remote depth estimation server to estimate depth from an image.

    Examples:
    >>> agent = DepthEstimationAgent(model_src="https://api.mbodi.ai/sense/")
    >>> result = agent.act(image=Image("resources/xarm.jpeg", size=(224, 224)))
    """

    def __init__(
        self,
        model_src="https://api.mbodi.ai/sense/",
        model_kwargs=None,
        **kwargs,
    ):
        super().__init__(
            model_src=model_src,
            model_kwargs=model_kwargs,
            **kwargs,
        )

    def act(self, image: Image, *args, api_name: str = "/depth", **kwargs) -> Image:
        """Act based on the prompt and image using the remote depth estimation server.

        Args:
            image (Image): The image to act on.
            *args: Variable length argument list.
            **kwargs: Arbitrary keyword arguments.

        Returns:
            Image: The depth image generated by the agent
        """
        if self.actor is None:
            raise ValueError("Remote actor for agent not initialized.")
        response = self.actor.predict(image.base64, *args, api_name=api_name, **kwargs)
        return Image(response)

act(image, *args, api_name='/depth', **kwargs)

Act based on the prompt and image using the remote depth estimation server.

Parameters:

Name Type Description Default
image Image

The image to act on.

 required
*args

Variable length argument list.

 ()
**kwargs

Arbitrary keyword arguments.

 {}

Returns:

Name Type Description
Image Image

The depth image generated by the agent
Source code in mbodied/agents/sense/depth_estimation_agent.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
def act(self, image: Image, *args, api_name: str = "/depth", **kwargs) -> Image:
    """Act based on the prompt and image using the remote depth estimation server.

    Args:
        image (Image): The image to act on.
        *args: Variable length argument list.
        **kwargs: Arbitrary keyword arguments.

    Returns:
        Image: The depth image generated by the agent
    """
    if self.actor is None:
        raise ValueError("Remote actor for agent not initialized.")
    response = self.actor.predict(image.base64, *args, api_name=api_name, **kwargs)
    return Image(response)