使用TensorDataset()和DataLoader()来构建数据集和dataloader

Handsome_zhou

在深度学习中，需要将数据分批的放入训练网络中，批数量的大小称为batch_size, 通过torch提供的dataloader方法，
可以实现一个迭代器。每次返回一组batch_size个样本和标签来训练。

1. from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler
   
2. 
   
3. #给训练集创建DataLoader
   
4. train_data = TensorDataset(train_inputs, train_masks, trian_labels)
   
5. train_sampler = RandomSampler(train_data)  # 训练集随机采样
   
6. train_dataloader = DataLoader(train_data, sampler = train_sampler, batch_size=batch_size)
   
7. 
   
8. #给验证集创建 DataLoader
   
9. valid_data = TensorDataset(test_inputs, test_masks, test_labels)
   
10. valid_sampler = SequentialSampler(valid_data)  # 验证集按同样的顺序采样
    
11. valid_dataloader = DataLoader(valid_data, sampler=valid_sampler, batch_size=batch_size)

复制代码