mmdet3d.apis¶

mmdet3d.datasets¶

datasets¶

transforms¶

mmdet3d.engine¶

hooks¶

mmdet3d.evaluation¶

functional¶

metrics¶

mmdet3d.models¶

backbones¶

data_preprocessors¶

decode_heads¶

dense_heads¶

detectors¶

layers¶

losses¶

middle_encoders¶

necks¶

roi_heads¶

segmentors¶

task_modules¶

test_time_augs¶

utils¶

voxel_encoders¶

mmdet3d.structures¶

structures¶

bbox_3d¶

class mmdet3d.structures.bbox_3d.BaseInstance3DBoxes(tensor, box_dim=7, with_yaw=True, origin=(0.5, 0.5, 0))[source]¶

Base class for 3D Boxes.

Note

The box is bottom centered, i.e. the relative position of origin in the box is (0.5, 0.5, 0).

Parameters

tensor (torch.Tensor | np.ndarray | list) – a N x box_dim matrix.
box_dim (int) – Number of the dimension of a box. Each row is (x, y, z, x_size, y_size, z_size, yaw). Defaults to 7.
with_yaw (bool) – Whether the box is with yaw rotation. If False, the value of yaw will be set to 0 as minmax boxes. Defaults to True.
origin (tuple[float], optional) – Relative position of the box origin. Defaults to (0.5, 0.5, 0). This will guide the box be converted to (0.5, 0.5, 0) mode.

tensor¶

Float matrix of N x box_dim.

Type: torch.Tensor

box_dim¶

Integer indicating the dimension of a box. Each row is (x, y, z, x_size, y_size, z_size, yaw, …).

Type: int

with_yaw¶

If True, the value of yaw will be set to 0 as minmax boxes.

Type: bool

property bev¶

2D BEV box of each box with rotation in XYWHR format, in shape (N, 5).

Type: torch.Tensor

property bottom_center¶

A tensor with center of each box in shape (N, 3).

Type: torch.Tensor

property bottom_height¶: torch.Tensor: A vector with bottom’s height of each box in shape (N, ).

classmethod cat(boxes_list)[source]¶

Concatenate a list of Boxes into a single Boxes.

Parameters: boxes_list (list[BaseInstance3DBoxes]) – List of boxes.
Returns: The concatenated Boxes.
Return type: BaseInstance3DBoxes

property center¶

Calculate the center of all the boxes.

Note

In MMDetection3D’s convention, the bottom center is usually taken as the default center.

The relative position of the centers in different kinds of boxes are different, e.g., the relative center of a boxes is (0.5, 1.0, 0.5) in camera and (0.5, 0.5, 0) in lidar. It is recommended to use bottom_center or gravity_center for clearer usage.

Returns: A tensor with center of each box in shape (N, 3).
Return type: torch.Tensor

clone()[source]¶

Clone the Boxes.

Returns

Box object with the same properties: as self.

Return type

BaseInstance3DBoxes

abstract convert_to(dst, rt_mat=None)[source]¶

Convert self to dst mode.

Parameters

dst (Box3DMode) – The target Box mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.

Returns

The converted box of the same type: in the dst mode.

Return type

BaseInstance3DBoxes

property corners¶: torch.Tensor: a tensor with 8 corners of each box in shape (N, 8, 3).

property device¶

The device of the boxes are on.

Type: str

property dims¶

Size dimensions of each box in shape (N, 3).

Type: torch.Tensor

abstract flip(bev_direction='horizontal')[source]¶

Flip the boxes in BEV along given BEV direction.

Parameters: bev_direction (str, optional) – Direction by which to flip. Can be chosen from ‘horizontal’ and ‘vertical’. Defaults to ‘horizontal’.

property gravity_center¶

A tensor with center of each box in shape (N, 3).

Type: torch.Tensor

property height¶

A vector with height of each box in shape (N, ).

Type: torch.Tensor

classmethod height_overlaps(boxes1, boxes2, mode='iou')[source]¶

Calculate height overlaps of two boxes.

Note

This function calculates the height overlaps between boxes1 and boxes2, boxes1 and boxes2 should be in the same type.

Parameters

boxes1 (BaseInstance3DBoxes) – Boxes 1 contain N boxes.
boxes2 (BaseInstance3DBoxes) – Boxes 2 contain M boxes.
mode (str, optional) – Mode of IoU calculation. Defaults to ‘iou’.

Returns

Calculated iou of boxes.

Return type

torch.Tensor

in_range_3d(box_range)[source]¶

Check whether the boxes are in the given range.

Parameters: box_range (list | torch.Tensor) – The range of box (x_min, y_min, z_min, x_max, y_max, z_max)

Note

In the original implementation of SECOND, checking whether a box in the range checks whether the points are in a convex polygon, we try to reduce the burden for simpler cases.

Returns

A binary vector indicating whether each box is: inside the reference range.

Return type

torch.Tensor

in_range_bev(box_range)[source]¶

Check whether the boxes are in the given range.

Parameters: box_range (list | torch.Tensor) – the range of box (x_min, y_min, x_max, y_max)

Note

The original implementation of SECOND checks whether boxes in a range by checking whether the points are in a convex polygon, we reduce the burden for simpler cases.

Returns: Whether each box is inside the reference range.
Return type: torch.Tensor

limit_yaw(offset=0.5, period=3.141592653589793)[source]¶

Limit the yaw to a given period and offset.

Parameters

offset (float, optional) – The offset of the yaw. Defaults to 0.5.
period (float, optional) – The expected period. Defaults to np.pi.

property nearest_bev¶

A tensor of 2D BEV box of each box without rotation.

Type: torch.Tensor

new_box(data)[source]¶

Create a new box object with data.

The new box and its tensor has the similar properties: as self and self.tensor, respectively.

Parameters

data (torch.Tensor | numpy.array | list) – Data to be copied.

Returns

A new bbox object with data,: the object’s other properties are similar to self.

Return type

BaseInstance3DBoxes

nonempty(threshold=0.0)[source]¶

Find boxes that are non-empty.

A box is considered empty, if either of its side is no larger than threshold.

Parameters

threshold (float, optional) – The threshold of minimal sizes. Defaults to 0.0.

Returns

A binary vector which represents whether each: box is empty (False) or non-empty (True).

Return type

torch.Tensor

classmethod overlaps(boxes1, boxes2, mode='iou')[source]¶

Calculate 3D overlaps of two boxes.

Note

This function calculates the overlaps between boxes1 and boxes2, boxes1 and boxes2 should be in the same type.

Parameters

boxes1 (BaseInstance3DBoxes) – Boxes 1 contain N boxes.
boxes2 (BaseInstance3DBoxes) – Boxes 2 contain M boxes.
mode (str, optional) – Mode of iou calculation. Defaults to ‘iou’.

Returns

Calculated 3D overlaps of the boxes.

Return type

torch.Tensor

points_in_boxes_all(points, boxes_override=None)[source]¶

Find all boxes in which each point is.

Parameters

points (torch.Tensor) – Points in shape (1, M, 3) or (M, 3), 3 dimensions are (x, y, z) in LiDAR or depth coordinate.
boxes_override (torch.Tensor, optional) – Boxes to override self.tensor. Defaults to None.

Returns

A tensor indicating whether a point is in a box,: in shape (M, T). T is the number of boxes. Denote this tensor as A, if the m^th point is in the t^th box, then A[m, t] == 1, elsewise A[m, t] == 0.

Return type

torch.Tensor

points_in_boxes_part(points, boxes_override=None)[source]¶

Find the box in which each point is.

Parameters

points (torch.Tensor) – Points in shape (1, M, 3) or (M, 3), 3 dimensions are (x, y, z) in LiDAR or depth coordinate.
boxes_override (torch.Tensor, optional) – Boxes to override self.tensor. Defaults to None.

Returns

The index of the first box that each point: is in, in shape (M, ). Default value is -1 (if the point is not enclosed by any box).

Return type

torch.Tensor

Note

If a point is enclosed by multiple boxes, the index of the first box will be returned.

abstract rotate(angle, points=None)[source]¶

Rotate boxes with points (optional) with the given angle or rotation matrix.

Parameters

angle (float | torch.Tensor | np.ndarray) – Rotation angle or rotation matrix.
(torch.Tensor | numpy.ndarray | (points) – BasePoints, optional): Points to rotate. Defaults to None.

scale(scale_factor)[source]¶

Scale the box with horizontal and vertical scaling factors.

Parameters: scale_factors (float) – Scale factors to scale the boxes.

to(device, *args, **kwargs)[source]¶

Convert current boxes to a specific device.

Parameters

device (str | torch.device) – The name of the device.

Returns

A new boxes object on the: specific device.

Return type

BaseInstance3DBoxes

property top_height¶: torch.Tensor: A vector with the top height of each box in shape (N, ).

translate(trans_vector)[source]¶

Translate boxes with the given translation vector.

Parameters: trans_vector (torch.Tensor) – Translation vector of size (1, 3).

property volume¶

A vector with volume of each box.

Type: torch.Tensor

property yaw¶

A vector with yaw of each box in shape (N, ).

Type: torch.Tensor

class mmdet3d.structures.bbox_3d.Box3DMode(value)[source]¶

Enum of different ways to represent a box.

Coordinates in LiDAR:

            up z
               ^   x front
               |  /
               | /
left y <------ 0

The relative coordinate of bottom center in a LiDAR box is (0.5, 0.5, 0), and the yaw is around the z axis, thus the rotation axis=2.

Coordinates in camera:

        z front
       /
      /
     0 ------> x right
     |
     |
     v
down y

The relative coordinate of bottom center in a CAM box is (0.5, 1.0, 0.5), and the yaw is around the y axis, thus the rotation axis=1.

Coordinates in Depth mode:

up z
   ^   y front
   |  /
   | /
   0 ------> x right

The relative coordinate of bottom center in a DEPTH box is (0.5, 0.5, 0), and the yaw is around the z axis, thus the rotation axis=2.

static convert(box, src, dst, rt_mat=None, with_yaw=True, correct_yaw=False)[source]¶

Convert boxes from src mode to dst mode.

Parameters

(tuple | list | np.ndarray | (box) – torch.Tensor | BaseInstance3DBoxes): Can be a k-tuple, k-list or an Nxk array/tensor, where k = 7.
src (Box3DMode) – The src Box mode.
dst (Box3DMode) – The target Box mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.
with_yaw (bool, optional) – If box is an instance of BaseInstance3DBoxes, whether or not it has a yaw angle. Defaults to True.
correct_yaw (bool) – If the yaw is rotated by rt_mat.

Returns

(tuple | list | np.ndarray | torch.Tensor |: BaseInstance3DBoxes): The converted box of the same type.

class mmdet3d.structures.bbox_3d.CameraInstance3DBoxes(tensor, box_dim=7, with_yaw=True, origin=(0.5, 1.0, 0.5))[source]¶

3D boxes of instances in CAM coordinates.

Coordinates in camera:

        z front (yaw=-0.5*pi)
       /
      /
     0 ------> x right (yaw=0)
     |
     |
     v
down y

The relative coordinate of bottom center in a CAM box is (0.5, 1.0, 0.5), and the yaw is around the y axis, thus the rotation axis=1. The yaw is 0 at the positive direction of x axis, and decreases from the positive direction of x to the positive direction of z.

tensor¶

Float matrix in shape (N, box_dim).

Type: torch.Tensor

box_dim¶

Integer indicating the dimension of a box Each row is (x, y, z, x_size, y_size, z_size, yaw, …).

Type: int

with_yaw¶

If True, the value of yaw will be set to 0 as axis-aligned boxes tightly enclosing the original boxes.

Type: bool

property bev¶

2D BEV box of each box with rotation in XYWHR format, in shape (N, 5).

Type: torch.Tensor

property bottom_height¶: torch.Tensor: A vector with bottom’s height of each box in shape (N, ).

convert_to(dst, rt_mat=None, correct_yaw=False)[source]¶

Convert self to dst mode.

Parameters

dst (Box3DMode) – The target Box mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.
correct_yaw (bool) – Whether to convert the yaw angle to the target coordinate. Defaults to False.

Returns

The converted box of the same type in the dst mode.

Return type

BaseInstance3DBoxes

property corners¶

Coordinates of corners of all the boxes in: shape (N, 8, 3).

Convert the boxes to in clockwise order, in the form of (x0y0z0, x0y0z1, x0y1z1, x0y1z0, x1y0z0, x1y0z1, x1y1z1, x1y1z0)

             front z
                  /
                 /
   (x0, y0, z1) + -----------  + (x1, y0, z1)
               /|            / |
              / |           /  |
(x0, y0, z0) + ----------- +   + (x1, y1, z1)
             |  /      .   |  /
             | / origin    | /
(x0, y1, z0) + ----------- + -------> x right
             |             (x1, y1, z0)
             |
             v
        down y

Type: torch.Tensor

flip(bev_direction='horizontal', points=None)[source]¶

Flip the boxes in BEV along given BEV direction.

In CAM coordinates, it flips the x (horizontal) or z (vertical) axis.

Parameters

bev_direction (str) – Flip direction (horizontal or vertical).
points (torch.Tensor | np.ndarray | BasePoints, optional) – Points to flip. Defaults to None.

Returns

Flipped points.

Return type

torch.Tensor, numpy.ndarray or None

property gravity_center¶

A tensor with center of each box in shape (N, 3).

Type: torch.Tensor

property height¶

A vector with height of each box in shape (N, ).

Type: torch.Tensor

classmethod height_overlaps(boxes1, boxes2, mode='iou')[source]¶

Calculate height overlaps of two boxes.

This function calculates the height overlaps between boxes1 and boxes2, where boxes1 and boxes2 should be in the same type.

Parameters

boxes1 (CameraInstance3DBoxes) – Boxes 1 contain N boxes.
boxes2 (CameraInstance3DBoxes) – Boxes 2 contain M boxes.
mode (str, optional) – Mode of iou calculation. Defaults to ‘iou’.

Returns

Calculated iou of boxes’ heights.

Return type

torch.Tensor

property local_yaw¶: torch.Tensor: A vector with local yaw of each box in shape (N, ). local_yaw equals to alpha in kitti, which is commonly used in monocular 3D object detection task, so only CameraInstance3DBoxes has the property.

points_in_boxes_all(points, boxes_override=None)[source]¶

Find all boxes in which each point is.

Parameters

points (torch.Tensor) – Points in shape (1, M, 3) or (M, 3), 3 dimensions are (x, y, z) in LiDAR or depth coordinate.
boxes_override (torch.Tensor, optional) – Boxes to override `self.tensor `. Defaults to None.

Returns

The index of all boxes in which each point is,: in shape (B, M, T).

Return type

torch.Tensor

points_in_boxes_part(points, boxes_override=None)[source]¶

Find the box in which each point is.

Parameters

points (torch.Tensor) – Points in shape (1, M, 3) or (M, 3), 3 dimensions are (x, y, z) in LiDAR or depth coordinate.
boxes_override (torch.Tensor, optional) – Boxes to override `self.tensor `. Defaults to None.

Returns

The index of the box in which: each point is, in shape (M, ). Default value is -1 (if the point is not enclosed by any box).

Return type

torch.Tensor

rotate(angle, points=None)[source]¶

Rotate boxes with points (optional) with the given angle or rotation matrix.

Parameters

angle (float | torch.Tensor | np.ndarray) – Rotation angle or rotation matrix.
points (torch.Tensor | np.ndarray | BasePoints, optional) – Points to rotate. Defaults to None.

Returns

When points is None, the function returns: None, otherwise it returns the rotated points and the rotation matrix rot_mat_T.

Return type

tuple or None

property top_height¶: torch.Tensor: A vector with the top height of each box in shape (N, ).

class mmdet3d.structures.bbox_3d.Coord3DMode(value)[source]¶

Enum of different ways to represent a box: and point cloud.

Coordinates in LiDAR:

            up z
               ^   x front
               |  /
               | /
left y <------ 0

The relative coordinate of bottom center in a LiDAR box is (0.5, 0.5, 0), and the yaw is around the z axis, thus the rotation axis=2.

Coordinates in camera:

        z front
       /
      /
     0 ------> x right
     |
     |
     v
down y

The relative coordinate of bottom center in a CAM box is (0.5, 1.0, 0.5), and the yaw is around the y axis, thus the rotation axis=1.

Coordinates in Depth mode:

up z
   ^   y front
   |  /
   | /
   0 ------> x right

The relative coordinate of bottom center in a DEPTH box is (0.5, 0.5, 0), and the yaw is around the z axis, thus the rotation axis=2.

static convert(input, src, dst, rt_mat=None, with_yaw=True, is_point=True)[source]¶

Convert boxes or points from src mode to dst mode.

Parameters

(tuple | list | np.ndarray | torch.Tensor | (input) – BaseInstance3DBoxes | BasePoints): Can be a k-tuple, k-list or an Nxk array/tensor, where k = 7.
src (Box3DMode | Coord3DMode) – The source mode.
dst (Box3DMode | Coord3DMode) – The target mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.
with_yaw (bool) – If box is an instance of BaseInstance3DBoxes, whether or not it has a yaw angle. Defaults to True.
is_point (bool) – If input is neither an instance of BaseInstance3DBoxes nor an instance of BasePoints, whether or not it is point data. Defaults to True.

Returns

(tuple | list | np.ndarray | torch.Tensor |: BaseInstance3DBoxes | BasePoints): The converted box of the same type.

static convert_box(box, src, dst, rt_mat=None, with_yaw=True)[source]¶

Convert boxes from src mode to dst mode.

Parameters

(tuple | list | np.ndarray | (box) – torch.Tensor | BaseInstance3DBoxes): Can be a k-tuple, k-list or an Nxk array/tensor, where k = 7.
src (Box3DMode) – The src Box mode.
dst (Box3DMode) – The target Box mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.
with_yaw (bool) – If box is an instance of BaseInstance3DBoxes, whether or not it has a yaw angle. Defaults to True.

Returns

(tuple | list | np.ndarray | torch.Tensor |: BaseInstance3DBoxes): The converted box of the same type.

static convert_point(point, src, dst, rt_mat=None)[source]¶

Convert points from src mode to dst mode.

Parameters

(tuple | list | np.ndarray | (point) – torch.Tensor | BasePoints): Can be a k-tuple, k-list or an Nxk array/tensor.
src (CoordMode) – The src Point mode.
dst (CoordMode) – The target Point mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.

Returns

The converted point of the same type.

Return type

(tuple | list | np.ndarray | torch.Tensor | BasePoints)

class mmdet3d.structures.bbox_3d.DepthInstance3DBoxes(tensor, box_dim=7, with_yaw=True, origin=(0.5, 0.5, 0))[source]¶

3D boxes of instances in Depth coordinates.

Coordinates in Depth:

up z    y front (yaw=0.5*pi)
   ^   ^
   |  /
   | /
   0 ------> x right (yaw=0)

The relative coordinate of bottom center in a Depth box is (0.5, 0.5, 0), and the yaw is around the z axis, thus the rotation axis=2. The yaw is 0 at the positive direction of x axis, and decreases from the positive direction of x to the positive direction of y. Also note that rotation of DepthInstance3DBoxes is counterclockwise, which is reverse to the definition of the yaw angle (clockwise).

A refactor is ongoing to make the three coordinate systems easier to understand and convert between each other.

tensor¶

Float matrix of N x box_dim.

Type: torch.Tensor

box_dim¶

Integer indicates the dimension of a box Each row is (x, y, z, x_size, y_size, z_size, yaw, …).

Type: int

with_yaw¶

If True, the value of yaw will be set to 0 as minmax boxes.

Type: bool

convert_to(dst, rt_mat=None)[source]¶

Convert self to dst mode.

Parameters

dst (Box3DMode) – The target Box mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.

Returns

The converted box of the same type in the dst mode.

Return type

DepthInstance3DBoxes

property corners¶

Coordinates of corners of all the boxes in shape (N, 8, 3).

Convert the boxes to corners in clockwise order, in form of (x0y0z0, x0y0z1, x0y1z1, x0y1z0, x1y0z0, x1y0z1, x1y1z1, x1y1z0)

                            up z
             front y           ^
                  /            |
                 /             |
   (x0, y1, z1) + -----------  + (x1, y1, z1)
               /|            / |
              / |           /  |
(x0, y0, z1) + ----------- +   + (x1, y1, z0)
             |  /      .   |  /
             | / origin    | /
(x0, y0, z0) + ----------- + --------> right x
                           (x1, y0, z0)

Type: torch.Tensor

enlarged_box(extra_width)[source]¶

Enlarge the length, width and height boxes.

Parameters: extra_width (float | torch.Tensor) – Extra width to enlarge the box.
Returns: Enlarged boxes.
Return type: DepthInstance3DBoxes

flip(bev_direction='horizontal', points=None)[source]¶

Flip the boxes in BEV along given BEV direction.

In Depth coordinates, it flips x (horizontal) or y (vertical) axis.

Parameters

bev_direction (str, optional) – Flip direction (horizontal or vertical). Defaults to ‘horizontal’.
points (torch.Tensor | np.ndarray | BasePoints, optional) – Points to flip. Defaults to None.

Returns

Flipped points.

Return type

torch.Tensor, numpy.ndarray or None

get_surface_line_center()[source]¶

Compute surface and line center of bounding boxes.

Returns: Surface and line center of bounding boxes.
Return type: torch.Tensor

property gravity_center¶

A tensor with center of each box in shape (N, 3).

Type: torch.Tensor

rotate(angle, points=None)[source]¶

Rotate boxes with points (optional) with the given angle or rotation matrix.

Parameters

angle (float | torch.Tensor | np.ndarray) – Rotation angle or rotation matrix.
points (torch.Tensor | np.ndarray | BasePoints, optional) – Points to rotate. Defaults to None.

Returns

When points is None, the function returns: None, otherwise it returns the rotated points and the rotation matrix rot_mat_T.

Return type

tuple or None

class mmdet3d.structures.bbox_3d.LiDARInstance3DBoxes(tensor, box_dim=7, with_yaw=True, origin=(0.5, 0.5, 0))[source]¶

3D boxes of instances in LIDAR coordinates.

Coordinates in LiDAR:

                         up z    x front (yaw=0)
                            ^   ^
                            |  /
                            | /
(yaw=0.5*pi) left y <------ 0

The relative coordinate of bottom center in a LiDAR box is (0.5, 0.5, 0), and the yaw is around the z axis, thus the rotation axis=2. The yaw is 0 at the positive direction of x axis, and increases from the positive direction of x to the positive direction of y.

A refactor is ongoing to make the three coordinate systems easier to understand and convert between each other.

tensor¶

Float matrix of N x box_dim.

Type: torch.Tensor

box_dim¶

Integer indicating the dimension of a box. Each row is (x, y, z, x_size, y_size, z_size, yaw, …).

Type: int

with_yaw¶

If True, the value of yaw will be set to 0 as minmax boxes.

Type: bool

convert_to(dst, rt_mat=None, correct_yaw=False)[source]¶

Convert self to dst mode.

Parameters

dst (Box3DMode) – the target Box mode
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.
correct_yaw (bool) – If convert the yaw angle to the target coordinate. Defaults to False.

Returns

The converted box of the same type in the dst mode.

Return type

BaseInstance3DBoxes

property corners¶

Coordinates of corners of all the boxes in shape (N, 8, 3).

Convert the boxes to corners in clockwise order, in form of (x0y0z0, x0y0z1, x0y1z1, x0y1z0, x1y0z0, x1y0z1, x1y1z1, x1y1z0)

                               up z
                front x           ^
                     /            |
                    /             |
      (x1, y0, z1) + -----------  + (x1, y1, z1)
                  /|            / |
                 / |           /  |
   (x0, y0, z1) + ----------- +   + (x1, y1, z0)
                |  /      .   |  /
                | / origin    | /
left y<-------- + ----------- + (x0, y1, z0)
    (x0, y0, z0)

Type: torch.Tensor

enlarged_box(extra_width)[source]¶

Enlarge the length, width and height boxes.

Parameters: extra_width (float | torch.Tensor) – Extra width to enlarge the box.
Returns: Enlarged boxes.
Return type: LiDARInstance3DBoxes

flip(bev_direction='horizontal', points=None)[source]¶

Flip the boxes in BEV along given BEV direction.

In LIDAR coordinates, it flips the y (horizontal) or x (vertical) axis.

Parameters

bev_direction (str) – Flip direction (horizontal or vertical).
points (torch.Tensor | np.ndarray | BasePoints, optional) – Points to flip. Defaults to None.

Returns

Flipped points.

Return type

torch.Tensor, numpy.ndarray or None

property gravity_center¶

A tensor with center of each box in shape (N, 3).

Type: torch.Tensor

rotate(angle, points=None)[source]¶

Rotate boxes with points (optional) with the given angle or rotation matrix.

Parameters

angles (float | torch.Tensor | np.ndarray) – Rotation angle or rotation matrix.
points (torch.Tensor | np.ndarray | BasePoints, optional) – Points to rotate. Defaults to None.

Returns

When points is None, the function returns: None, otherwise it returns the rotated points and the rotation matrix rot_mat_T.

Return type

tuple or None

mmdet3d.structures.bbox_3d.get_box_type(box_type)[source]¶

Get the type and mode of box structure.

Parameters: box_type (str) – The type of box structure. The valid value are “LiDAR”, “Camera”, or “Depth”.
Raises: ValueError – A ValueError is raised when box_type does not belong to the three valid types.
Returns: Box type and box mode.
Return type: tuple

mmdet3d.structures.bbox_3d.get_proj_mat_by_coord_type(img_meta, coord_type)[source]¶

Obtain image features using points.

Parameters

img_meta (dict) – Meta info.
coord_type (str) – ‘DEPTH’ or ‘CAMERA’ or ‘LIDAR’. Can be case-insensitive.

Returns

transformation matrix.

Return type

torch.Tensor

mmdet3d.structures.bbox_3d.limit_period(val, offset=0.5, period=3.141592653589793)[source]¶

Limit the value into a period for periodic function.

Parameters

val (torch.Tensor | np.ndarray) – The value to be converted.
offset (float, optional) – Offset to set the value range. Defaults to 0.5.
period ([type], optional) – Period of the value. Defaults to np.pi.

Returns

Value in the range of: [-offset * period, (1-offset) * period]

Return type

(torch.Tensor | np.ndarray)

mmdet3d.structures.bbox_3d.mono_cam_box2vis(cam_box)[source]¶

This is a post-processing function on the bboxes from Mono-3D task. If we want to perform projection visualization, we need to:

rotate the box along x-axis for np.pi / 2 (roll)

change orientation from local yaw to global yaw

convert yaw by (np.pi / 2 - yaw)

After applying this function, we can project and draw it on 2D images.

Parameters: cam_box (CameraInstance3DBoxes) – 3D bbox in camera coordinate system before conversion. Could be gt bbox loaded from dataset or network prediction output.
Returns: Box after conversion.
Return type: CameraInstance3DBoxes

mmdet3d.structures.bbox_3d.points_cam2img(points_3d, proj_mat, with_depth=False)[source]¶

Project points in camera coordinates to image coordinates.

Parameters

points_3d (torch.Tensor | np.ndarray) – Points in shape (N, 3)
proj_mat (torch.Tensor | np.ndarray) – Transformation matrix between coordinates.
with_depth (bool, optional) – Whether to keep depth in the output. Defaults to False.

Returns

Points in image coordinates,: with shape [N, 2] if with_depth=False, else [N, 3].

Return type

(torch.Tensor | np.ndarray)

mmdet3d.structures.bbox_3d.points_img2cam(points, cam2img)[source]¶

Project points in image coordinates to camera coordinates.

Parameters

points (torch.Tensor) – 2.5D points in 2D images, [N, 3], 3 corresponds with x, y in the image and depth.
cam2img (torch.Tensor) – Camera intrinsic matrix. The shape can be [3, 3], [3, 4] or [4, 4].

Returns

points in 3D space. [N, 3],: 3 corresponds with x, y, z in 3D space.

Return type

torch.Tensor

mmdet3d.structures.bbox_3d.rotation_3d_in_axis(points, angles, axis=0, return_mat=False, clockwise=False)[source]¶

Rotate points by angles according to axis.

Parameters

points (np.ndarray | torch.Tensor | list | tuple) – Points of shape (N, M, 3).
angles (np.ndarray | torch.Tensor | list | tuple | float) – Vector of angles in shape (N,)
axis (int, optional) – The axis to be rotated. Defaults to 0.
return_mat – Whether or not return the rotation matrix (transposed). Defaults to False.
clockwise – Whether the rotation is clockwise. Defaults to False.

Raises

ValueError – when the axis is not in range [0, 1, 2], it will raise value error.

Returns

Rotated points in shape (N, M, 3).

Return type

(torch.Tensor | np.ndarray)

mmdet3d.structures.bbox_3d.xywhr2xyxyr(boxes_xywhr)[source]¶

Convert a rotated boxes in XYWHR format to XYXYR format.

Parameters: boxes_xywhr (torch.Tensor | np.ndarray) – Rotated boxes in XYWHR format.
Returns: Converted boxes in XYXYR format.
Return type: (torch.Tensor | np.ndarray)

ops¶

points¶

class mmdet3d.structures.points.BasePoints(tensor, points_dim=3, attribute_dims=None)[source]¶

Base class for Points.

Parameters

tensor (torch.Tensor | np.ndarray | list) – a N x points_dim matrix.
points_dim (int, optional) – Number of the dimension of a point. Each row is (x, y, z). Defaults to 3.
attribute_dims (dict, optional) – Dictionary to indicate the meaning of extra dimension. Defaults to None.

tensor¶

Float matrix of N x points_dim.

Type: torch.Tensor

points_dim¶

Integer indicating the dimension of a point. Each row is (x, y, z, …).

Type: int

attribute_dims¶

Dictionary to indicate the meaning of extra dimension. Defaults to None.

Type: bool

rotation_axis¶

Default rotation axis for points rotation.

Type: int

property bev¶

BEV of the points in shape (N, 2).

Type: torch.Tensor

classmethod cat(points_list)[source]¶

Concatenate a list of Points into a single Points.

Parameters: points_list (list[BasePoints]) – List of points.
Returns: The concatenated Points.
Return type: BasePoints

clone()[source]¶

Clone the Points.

Returns

Box object with the same properties: as self.

Return type

BasePoints

property color¶: torch.Tensor: A vector with color of each point in shape (N, 3), or None.

abstract convert_to(dst, rt_mat=None)[source]¶

Convert self to dst mode.

Parameters

dst (CoordMode) – The target Box mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.

Returns

The converted box of the same type: in the dst mode.

Return type

BasePoints

property coord¶

Coordinates of each point in shape (N, 3).

Type: torch.Tensor

property device¶

The device of the points are on.

Type: str

abstract flip(bev_direction='horizontal')[source]¶

Flip the points along given BEV direction.

Parameters: bev_direction (str) – Flip direction (horizontal or vertical).

property height¶: torch.Tensor: A vector with height of each point in shape (N, 1), or None.

in_range_3d(point_range)[source]¶

Check whether the points are in the given range.

Parameters: point_range (list | torch.Tensor) – The range of point (x_min, y_min, z_min, x_max, y_max, z_max)

Note

In the original implementation of SECOND, checking whether a box in the range checks whether the points are in a convex polygon, we try to reduce the burden for simpler cases.

Returns

A binary vector indicating whether each point is: inside the reference range.

Return type

torch.Tensor

in_range_bev(point_range)[source]¶

Check whether the points are in the given range.

Parameters

point_range (list | torch.Tensor) – The range of point in order of (x_min, y_min, x_max, y_max).

Returns

Indicating whether each point is inside: the reference range.

Return type

torch.Tensor

new_point(data)[source]¶

Create a new point object with data.

The new point and its tensor has the similar properties: as self and self.tensor, respectively.

Parameters

data (torch.Tensor | numpy.array | list) – Data to be copied.

Returns

A new point object with data,: the object’s other properties are similar to self.

Return type

BasePoints

rotate(rotation, axis=None)[source]¶

Rotate points with the given rotation matrix or angle.

Parameters

rotation (float | np.ndarray | torch.Tensor) – Rotation matrix or angle.
axis (int, optional) – Axis to rotate at. Defaults to None.

scale(scale_factor)[source]¶

Scale the points with horizontal and vertical scaling factors.

Parameters: scale_factors (float) – Scale factors to scale the points.

property shape¶

Shape of points.

Type: torch.Shape

shuffle()[source]¶

Shuffle the points.

Returns: The shuffled index.
Return type: torch.Tensor

to(device)[source]¶

Convert current points to a specific device.

Parameters

device (str | torch.device) – The name of the device.

Returns

A new boxes object on the: specific device.

Return type

BasePoints

translate(trans_vector)[source]¶

Translate points with the given translation vector.

Parameters: trans_vector (np.ndarray, torch.Tensor) – Translation vector of size 3 or nx3.

class mmdet3d.structures.points.CameraPoints(tensor, points_dim=3, attribute_dims=None)[source]¶

Points of instances in CAM coordinates.

Parameters

tensor (torch.Tensor | np.ndarray | list) – a N x points_dim matrix.
points_dim (int, optional) – Number of the dimension of a point. Each row is (x, y, z). Defaults to 3.
attribute_dims (dict, optional) – Dictionary to indicate the meaning of extra dimension. Defaults to None.

tensor¶

Float matrix of N x points_dim.

Type: torch.Tensor

points_dim¶

Integer indicating the dimension of a point. Each row is (x, y, z, …).

Type: int

attribute_dims¶

Dictionary to indicate the meaning of extra dimension. Defaults to None.

Type: bool

rotation_axis¶

Default rotation axis for points rotation.

Type: int

property bev¶

BEV of the points in shape (N, 2).

Type: torch.Tensor

convert_to(dst, rt_mat=None)[source]¶

Convert self to dst mode.

Parameters

dst (CoordMode) – The target Point mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.

Returns

The converted point of the same type: in the dst mode.

Return type

BasePoints

flip(bev_direction='horizontal')[source]¶

Flip the points along given BEV direction.

Parameters: bev_direction (str) – Flip direction (horizontal or vertical).

class mmdet3d.structures.points.DepthPoints(tensor, points_dim=3, attribute_dims=None)[source]¶

Points of instances in DEPTH coordinates.

Parameters

tensor (torch.Tensor | np.ndarray | list) – a N x points_dim matrix.
points_dim (int, optional) – Number of the dimension of a point. Each row is (x, y, z). Defaults to 3.
attribute_dims (dict, optional) – Dictionary to indicate the meaning of extra dimension. Defaults to None.

tensor¶

Float matrix of N x points_dim.

Type: torch.Tensor

points_dim¶

Integer indicating the dimension of a point. Each row is (x, y, z, …).

Type: int

attribute_dims¶

Dictionary to indicate the meaning of extra dimension. Defaults to None.

Type: bool

rotation_axis¶

Default rotation axis for points rotation.

Type: int

convert_to(dst, rt_mat=None)[source]¶

Convert self to dst mode.

Parameters

dst (CoordMode) – The target Point mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.

Returns

The converted point of the same type: in the dst mode.

Return type

BasePoints

flip(bev_direction='horizontal')[source]¶

Flip the points along given BEV direction.

Parameters: bev_direction (str) – Flip direction (horizontal or vertical).

class mmdet3d.structures.points.LiDARPoints(tensor, points_dim=3, attribute_dims=None)[source]¶

Points of instances in LIDAR coordinates.

Parameters

tensor (torch.Tensor | np.ndarray | list) – a N x points_dim matrix.
points_dim (int, optional) – Number of the dimension of a point. Each row is (x, y, z). Defaults to 3.
attribute_dims (dict, optional) – Dictionary to indicate the meaning of extra dimension. Defaults to None.

tensor¶

Float matrix of N x points_dim.

Type: torch.Tensor

points_dim¶

Integer indicating the dimension of a point. Each row is (x, y, z, …).

Type: int

attribute_dims¶

Dictionary to indicate the meaning of extra dimension. Defaults to None.

Type: bool

rotation_axis¶

Default rotation axis for points rotation.

Type: int

convert_to(dst, rt_mat=None)[source]¶

Convert self to dst mode.

Parameters

dst (CoordMode) – The target Point mode.
rt_mat (np.ndarray | torch.Tensor, optional) – The rotation and translation matrix between different coordinates. Defaults to None. The conversion from src coordinates to dst coordinates usually comes along the change of sensors, e.g., from camera to LiDAR. This requires a transformation matrix.

Returns

The converted point of the same type: in the dst mode.

Return type

BasePoints

flip(bev_direction='horizontal')[source]¶

Flip the points along given BEV direction.

Parameters: bev_direction (str) – Flip direction (horizontal or vertical).

mmdet3d.testing¶

mmdet3d.visualization¶

mmdet3d.utils¶

class mmdet3d.utils.ArrayConverter(template_array: Optional[Union[tuple, list, int, float, numpy.ndarray, torch.Tensor]] = None)[source]¶

Utility class for data-type agnostic processing.

Parameters: (tuple | list | int | float | np.ndarray | (template_array) – torch.Tensor, optional): template array. Defaults to None.

convert(input_array: Union[tuple, list, int, float, numpy.ndarray, torch.Tensor], target_type: Optional[type] = None, target_array: Optional[Union[numpy.ndarray, torch.Tensor]] = None) → Union[numpy.ndarray, torch.Tensor][source]¶

Convert input array to target data type.

Parameters: (tuple | list | int | float | np.ndarray | (input_array) – torch.Tensor): Input array.

:param target_type (np.ndarray or torch.Tensor: optional): Type to which input array is converted.: Defaults to None.
:paramoptional): Type to which input array is converted.: Defaults to None.

Parameters

target_array (np.ndarray | torch.Tensor, optional) – Template array to which input array is converted. Defaults to None.

Raises

ValueError – If input is list or tuple and cannot be converted to to a NumPy array, a ValueError is raised.
TypeError – If input type does not belong to the above range, or the contents of a list or tuple do not share the same data type, a TypeError is raised.

Returns

The converted array.

Return type

np.ndarray or torch.Tensor

recover(input_array: Union[numpy.ndarray, torch.Tensor]) → Union[numpy.ndarray, torch.Tensor][source]¶

Recover input type to original array type.

Parameters: input_array (np.ndarray | torch.Tensor) – Input array.
Returns: Converted array.
Return type: np.ndarray or torch.Tensor

set_template(array: Union[tuple, list, int, float, numpy.ndarray, torch.Tensor]) → None[source]¶

Set template array.

Parameters

Raises

ValueError – If input is list or tuple and cannot be converted to to a NumPy array, a ValueError is raised.
TypeError – If input type does not belong to the above range, or the contents of a list or tuple do not share the same data type, a TypeError is raised.

mmdet3d.utils.array_converter(to_torch: bool = True, apply_to: Tuple[str, ...] = (), template_arg_name_: Optional[str] = None, recover: bool = True) → Callable[source]¶

Wrapper function for data-type agnostic processing.

First converts input arrays to PyTorch tensors or NumPy ndarrays for middle calculation, then convert output to original data-type if recover=True.

Parameters

to_torch (bool) – Whether convert to PyTorch tensors for middle calculation. Defaults to True.
apply_to (Tuple[str, ...]) – The arguments to which we apply data-type conversion. Defaults to an empty tuple.
template_arg_name (str, optional) – Argument serving as the template ( return arrays should have the same dtype and device as the template). Defaults to None. If None, we will use the first argument in apply_to as the template argument.
recover (bool) – Whether or not recover the wrapped function outputs to the template_arg_name_ type. Defaults to True.

Raises

ValueError – When template_arg_name_ is not among all args, or when apply_to contains an arg which is not among all args, a ValueError will be raised. When the template argument or an argument to convert is a list or tuple, and cannot be converted to a NumPy array, a ValueError will be raised.
TypeError – When the type of the template argument or an argument to convert does not belong to the above range, or the contents of such an list-or-tuple-type argument do not share the same data type, a TypeError is raised.

Returns

wrapped function.

Return type

(function)

Example

>>> import torch
>>> import numpy as np
>>>
>>> # Use torch addition for a + b,
>>> # and convert return values to the type of a
>>> @array_converter(apply_to=('a', 'b'))
>>> def simple_add(a, b):
>>>     return a + b
>>>
>>> a = np.array([1.1])
>>> b = np.array([2.2])
>>> simple_add(a, b)
>>>
>>> # Use numpy addition for a + b,
>>> # and convert return values to the type of b
>>> @array_converter(to_torch=False, apply_to=('a', 'b'),
>>>                  template_arg_name_='b')
>>> def simple_add(a, b):
>>>     return a + b
>>>
>>> simple_add()
>>>
>>> # Use torch funcs for floor(a) if flag=True else ceil(a),
>>> # and return the torch tensor
>>> @array_converter(apply_to=('a',), recover=False)
>>> def floor_or_ceil(a, flag=True):
>>>     return torch.floor(a) if flag else torch.ceil(a)
>>>
>>> floor_or_ceil(a, flag=False)

mmdet3d.utils.collect_env()[source]¶: Collect the information of the running environments.

mmdet3d.utils.compat_cfg(cfg)[source]¶

This function would modify some filed to keep the compatibility of config.

For example, it will move some args which will be deprecated to the correct fields.

mmdet3d.utils.register_all_modules(init_default_scope: bool = True) → None[source]¶

Register all modules in mmdet3d into the registries.

Parameters: init_default_scope (bool) – Whether initialize the mmdet default scope. When init_default_scope=True, the global default scope will be set to mmdet3d, and all registries will build modules from mmdet3d’s registry node. To understand more about the registry, please refer to https://github.com/open-mmlab/mmengine/blob/main/docs/en/tutorials/registry.md Defaults to True.

mmdet3d.utils.setup_multi_processes(cfg)[source]¶: Setup multi-processing environment variables.