。

包括环境准备、下载数据集、数据采样、数据集加载和预处理、构建模型、损失函数、模型训练、模型评估等。

一、概念

AI框架使用SSD目标检测算法对图像内容识别的过程、步骤和方法。

包括环境准备、下载数据集、数据采样、数据集加载和预处理、构建模型、损失函数、模型训练、模型评估等。

一、概念

实验环境已经预装了mindspore2.2.14如需更换mindspore版本可更改下面mindspore的版本号

!pip

https://pypi.mirrors.ustc.edu.cn/simple

mindspore2.2.14

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

Requires:

mindspore、download、pycocotools、opencv-python

!pip

https://pypi.mirrors.ustc.edu.cn/simple

pycocotools2.0.7

https://pypi.mirrors.ustc.edu.cn/simple

Collecting

https://mirrors.bfsu.edu.cn/pypi/web/packages/19/93/5aaec888e3aa4d05b3a1472f331b83f7dc684d9a6b2645709d8f3352ba00/pycocotools-2.0.7-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

(419

kB)━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

419.9/419.9

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(1.2.1)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(0.12.1)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(4.53.0)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(1.4.5)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(23.2)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(10.3.0)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(3.1.2)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(2.9.0.post0)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

matplotlib2.1.0-pycocotools2.0.7)

(6.4.0)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

importlib-resources3.2.0-matplotlib2.1.0-pycocotools2.0.7)

(3.17.0)

/home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages

(from

python-dateutil2.7-matplotlib2.1.0-pycocotools2.0.7)

(1.16.0)

https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/ssd_datasets.zip

path

https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/ssd_datasets.zip

(16.0

100%|███████████████████████████|

16.8M/16.8M

./datasets/annotations/instances_val2017.json

train_cls

enumerate(train_cls):train_cls_dict[cls]

2.数据采样

采样区域和原始图片最小的交并比重叠为0.1,0.3,0.5,0.7或0.9

随机采样一个区域

ssd_bboxes_encode(boxes):Labels

anchors

jaccard_with_anchors(bbox):Compute

jaccard

default_boxes[index]bboxes_t[:,

:2]

np.array([len(np.nonzero(t_label)[0])],

dtypenp.int32)return

dataset.cv2.setNumThreads(2)def

_infer_data(image,

create_ssd_dataset(mindrecord_file,

batch_size32,

use_multiprocessingTrue):Create

SSD

num_shardsdevice_num,shard_idrank,

num_parallel_workersnum_parallel_workers,

Decode()dataset

input_columns[image])change_swap_op

HWC2CHW()#

RandomColorAdjust(brightness0.4,

contrast0.4,

change_swap_op]else:output_columns

[img_id,

dataset.map(operationscompose_map_func,

image,

annotation],output_columnsoutput_columns,

python_multiprocessinguse_multiprocessing,num_parallel_workersnum_parallel_workers)dataset

python_multiprocessinguse_multiprocessing,num_parallel_workersnum_parallel_workers)dataset

dataset

SSD使用第4、7、8、9、10和11这6个卷积层得到的特征图

6个特征图上的每一个点分别对应4、6、6、6、4、4个PriorBox。

利用特征图的特征预测每个PriorBox对应类别与位置的预测量

定义MultiBox类

_make_layer(channels):in_channels

channels[0]layers

channels[1:]:layers.append(nn.Conv2d(in_channelsin_channels,

kernel_size3))layers.append(nn.ReLU())in_channels

out_channelsreturn

nn.SequentialCell([depthwise_conv,

bn,

FlattenConcat(nn.Cell):FlattenConcat

module.def

__init__(self):super(FlattenConcat,

self).__init__()self.num_ssd_boxes

8732def

enumerate(out_channels):loc_layers

stride1,

nn.CellList(loc_layers)self.multi_cls_layers

nn.CellList(cls_layers)self.flatten_concat

FlattenConcat()def

range(len(self.multi_loc_layers)):loc_outputs

(self.multi_loc_layers[i](inputs[i]),)cls_outputs

(self.multi_cls_layers[i](inputs[i]),)return

self.flatten_concat(loc_outputs),

self.flatten_concat(cls_outputs)class

SSD300Vgg16(nn.Cell):SSD300Vgg16

module.def

__init__(self):super(SSD300Vgg16,

self).__init__()#

self.multi_box(multi_feature)if

not

ops.sigmoid(pred_label)pred_loc

pred_loc.astype(ms.float32)pred_label

pred_label.astype(ms.float32)return

pred_loc,

ops.ones_like(logits)pos_weight

ops.ones_like(logits)sigmiod_cross_entropy

ops.binary_cross_entropy_with_logits(logits,

label,

pos_weight.astype(ms.float32))sigmoid

label

apply_eval(eval_param_dict):net

eval_param_dict[net]net.set_train(False)ds

eval_param_dict[dataset]anno_json

eval_param_dict[anno_json]coco_metrics

COCOMetrics(anno_jsonanno_json,classestrain_cls,num_classes81,max_boxes100,nms_threshold0.6,min_score0.1)for

data

ds.create_dict_iterator(output_numpyTrue,

num_epochs1):img_id

range(img_np.shape[0]):pred_batch

{boxes:

output[0].asnumpy()[batch_idx],box_scores:

output[1].asnumpy()[batch_idx],img_id:

int(np.squeeze(img_id[batch_idx])),image_shape:

image_shape[batch_idx]}coco_metrics.update(pred_batch)eval_metrics

coco_metrics.get_metrics()return

eval_metricsdef

enumerate(classes)}self.coco_gt

self.coco_gt.loadCats(self.coco_gt.getCatIds())self.class_dict

{cat[name]:

[]self.img_ids.append(img_id)for

range(1,

self.num_classes):class_box_scores

box_scores[:,

class_box_scores[score_mask]class_boxes

[h,

class_boxes[nms_index]class_box_scores

class_box_scores[nms_index]final_boxes

class_boxes.tolist()final_score

class_box_scores.tolist()final_label

[self.class_dict[self.val_cls_dict[c]]]

loc,

labelself.predictions.append(res)def

f)coco_dt

self.coco_gt.loadRes(predictions.json)E

coco_dt,

self.img_idsE.evaluate()E.accumulate()E.summarize()return

E.stats[0]class

ckpt_path):super(SsdInferWithDecoder,

ms.load_checkpoint(ckpt_path)ms.load_param_into_net(network,

default_boxesself.prior_scaling_xy

0.2def

archor_sizes).self.default_boxes

has

math.sqrt(aspect_ratio)all_sizes.append((w,

sk3))

it.product(range(feature_size),

repeat2):for

fk[idex]self.default_boxes.append([cy,

cx,

calculationself.default_boxes_tlbr

for

dtypefloat32)self.default_boxes

dtypefloat32)

GeneratDefaultBoxes().default_boxes_tlbr

default_boxes

GeneratDefaultBoxes().default_boxes

y1,

initialize_modeTruncatedNormal):Init

***

TruncatedNormal:p.set_data(initializer(TruncatedNormal(0.02),

p.data.shape,

p.data.dtype))else:p.set_data(initialize_mode,

p.data.shape,

np.array(lr_each_step).astype(np.float32)learning_rate

​return

ds.config.set_enable_shared_mem(False)

import

./datasets/MindRecord_COCO/ssd.mindrecord0

dataset

create_ssd_dataset(mindrecord_file,

batch_size5,

next(dataset.create_tuple_iterator())

Network

gt_label).astype(ms.float32)num_matched_boxes

ops.sum(num_matched_boxes.astype(ms.float32))

Positioning

num_matched_boxes)opt(grads)return

loss

range(60):network.set_train(True)begin_time

time.time()for

enumerate(dataset.create_tuple_iterator()):loss

train_step(image,

begin_timeprint(fEpoch:[{int(epoch

1)}/{int(60)}],

调用SsdInferWithDecoder类返回预测的坐标及标签

Average

./datasets/MindRecord_COCO/ssd_eval.mindrecord0

def

create_ssd_dataset(dataset_path,

batch_sizebatch_size,is_trainingFalse,

​network

batch_sizeprint(\n\n)print(total

images

apply_eval(eval_param_dict)print(\n\n)print(fmAP:

{mAP})

Eval!)ssd_eval(mindrecord_file,

./ssd-60_9.ckpt,