译(五十四)-RuntimeError Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

stackoverflow热门问题目录

如有翻译问题欢迎评论指出,谢谢。

第三个回答看得我好懵,但我也不确定是不是真的有这种情况…

有个类似的报错见这里:Pytorch数据使用列表的卷积层时报错及解决-RuntimeError: Input type (torch.cuda.HalfTensor) and weight type (torch.FloatTensor) should be same

RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

  • Liz asked:

    • 我在用下面的代码训练一个 CNN 时,出现了关于 .cuda() 的报错,怎么解决?下面是涉及到的部分代码。

    • import matplotlib.pyplot as plt
      import numpy as np
      import torch
      from torch import nn
      from torch import optim
      import torch.nn.functional as F
      import torchvision
      from torchvision import datasets, transforms, models
      from torch.utils.data.sampler import SubsetRandomSampler
      #
      #
      data_dir = "/home/ubuntu/ML2/ExamII/train2/"
      valid_size = .2
      #
      # Normalize the test and train sets with torchvision
      train_transforms = transforms.Compose([transforms.Resize(224),
                                             transforms.ToTensor(),
                                             ])
      #
      test_transforms = transforms.Compose([transforms.Resize(224),
                                            transforms.ToTensor(),
                                            ])
      #
      # ImageFolder class to load the train and test images
      train_data = datasets.ImageFolder(data_dir, transform=train_transforms)
      test_data = datasets.ImageFolder(data_dir, transform=test_transforms)
      #
      #
      # Number of train images
      num_train = len(train_data)
      indices = list(range(num_train))
      # Split = 20% of train images
      split = int(np.floor(valid_size * num_train))
      # Shuffle indices of train images
      np.random.shuffle(indices)
      # Subset indices for test and train
      train_idx, test_idx = indices[split:], indices[:split]
      # Samples elements randomly from a given list of indices
      train_sampler = SubsetRandomSampler(train_idx)
      test_sampler = SubsetRandomSampler(test_idx)
      # Batch and load the images
      trainloader = torch.utils.data.DataLoader(train_data, sampler=train_sampler, batch_size=1)
      testloader = torch.utils.data.DataLoader(test_data, sampler=test_sampler, batch_size=1)
      #
      #
      #print(trainloader.dataset.classes)
      #
      device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
      model = models.resnet50(pretrained=True)
      #
      model.fc = nn.Sequential(nn.Linear(2048, 512),
                                   nn.ReLU(),
                                   nn.Dropout(0.2),
                                   nn.Linear(512, 10),
                                   nn.LogSigmoid())
                                   # nn.LogSoftmax(dim=1))
      # criterion = nn.NLLLoss()
      criterion = nn.BCELoss()
      optimizer = optim.Adam(model.fc.parameters(), lr=0.003)
      model.to(device)
      #
      #Train the network
      for epoch in range(2):  # loop over the dataset multiple times
      #
      running_loss = 0.0
      for i, data in enumerate(trainloader, 0):
          # get the inputs; data is a list of [inputs, labels]
          inputs, labels = data
          #
          # zero the parameter gradients
          optimizer.zero_grad()
          #
          # forward + backward + optimize
          outputs = model(inputs)
          loss = criterion(outputs, labels)
          loss.backward()
          optimizer.step()
          #
          # print statistics
          running_loss += loss.item()
          if i % 2000 == 1999:    # print every 2000 mini-batches
              print('[%d, %5d] loss: %.3f' %
                    (epoch + 1, i + 1, running_loss / 2000))
              running_loss = 0.0
      #
      print('Finished Training')
    • 然而,控制台始终报错:

    RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same`

    • 怎么解决?我感觉应该是模型没有送到 GPU 里,但不太清楚具体怎么解决。
  • Answers:

    • Nicolas Gervais – vote: 172

    • 这个报错是因为你的模型在 GPU,而数据在 CPU。因此需要把输入的张量送到 GPU。

    • inputs, labels = data                         # this is what you had
      inputs, labels = inputs.cuda(), labels.cuda() # add this line
    • 或者这样,和其它代码保持一致:

    • device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
      inputs, labels = inputs.to(device), labels.to(device)
    • 如果输入张量在 GPU,而模型权重不是的时候,还会出现同样的错误。这时需要把模型权重送入 GPU。

    • model = MyModel()
      if torch.cuda.is_available():
        model.cuda()
    • 见文档: cuda()cpu()

    • prosti – vote: 20

    • 有个新的 API,使用的 .to() 方法。

    • 这样做很明显更方便,说不定明天你的设备就不再是 cuda,而是下面的其它设备了:

      • cpu
      • cuda
      • mkldnn
      • opengl
      • opencl
      • ideep
      • hip
      • msnpu
      • xla
    • 所以尽量避免 model.cuda()

    • dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
    • 或这样:

    • dev=torch.device("cuda") 
    • 或这样:

    • dev="cuda"
    • 之后这样送入设备就行:

    • model.to(dev)
      data = data.to(dev)
    • tsveti_iko – vote: 2

    • 正如前面其它回答提到的,这是由于模型训练在 GPU,测试在 CPU 导致的。因此需要将模型权重与数据从 GPU 移向 CPU:

    • device = args.device # "cuda" / "cpu"
      if "cuda" in device and not torch.cuda.is_available():
        device = "cpu"
      data = data.to(device)
      model.to(device)
    • 注:这里仍需核对 GPU/CPU 的配置参数,以便代码可以用于在 GPU 训练,在 CPU 测试。


RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

  • Liz asked:

    • I am trying to train the following CNN as follows, but I keep getting the same error regarding .cuda() and I am not sure how to fix it. Here is a chunk of my code so far.
      我在用下面的代码训练一个 CNN 时,出现了关于 .cuda() 的报错,怎么解决?下面是涉及到的部分代码。

    • import matplotlib.pyplot as plt
      import numpy as np
      import torch
      from torch import nn
      from torch import optim
      import torch.nn.functional as F
      import torchvision
      from torchvision import datasets, transforms, models
      from torch.utils.data.sampler import SubsetRandomSampler
      #
      #
      data_dir = "/home/ubuntu/ML2/ExamII/train2/"
      valid_size = .2
      #
      # Normalize the test and train sets with torchvision
      train_transforms = transforms.Compose([transforms.Resize(224),
                                             transforms.ToTensor(),
                                             ])
      #
      test_transforms = transforms.Compose([transforms.Resize(224),
                                            transforms.ToTensor(),
                                            ])
      #
      # ImageFolder class to load the train and test images
      train_data = datasets.ImageFolder(data_dir, transform=train_transforms)
      test_data = datasets.ImageFolder(data_dir, transform=test_transforms)
      #
      #
      # Number of train images
      num_train = len(train_data)
      indices = list(range(num_train))
      # Split = 20% of train images
      split = int(np.floor(valid_size * num_train))
      # Shuffle indices of train images
      np.random.shuffle(indices)
      # Subset indices for test and train
      train_idx, test_idx = indices[split:], indices[:split]
      # Samples elements randomly from a given list of indices
      train_sampler = SubsetRandomSampler(train_idx)
      test_sampler = SubsetRandomSampler(test_idx)
      # Batch and load the images
      trainloader = torch.utils.data.DataLoader(train_data, sampler=train_sampler, batch_size=1)
      testloader = torch.utils.data.DataLoader(test_data, sampler=test_sampler, batch_size=1)
      #
      #
      #print(trainloader.dataset.classes)
      #
      device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
      model = models.resnet50(pretrained=True)
      #
      model.fc = nn.Sequential(nn.Linear(2048, 512),
                                   nn.ReLU(),
                                   nn.Dropout(0.2),
                                   nn.Linear(512, 10),
                                   nn.LogSigmoid())
                                   # nn.LogSoftmax(dim=1))
      # criterion = nn.NLLLoss()
      criterion = nn.BCELoss()
      optimizer = optim.Adam(model.fc.parameters(), lr=0.003)
      model.to(device)
      #
      #Train the network
      for epoch in range(2):  # loop over the dataset multiple times
      #
      running_loss = 0.0
      for i, data in enumerate(trainloader, 0):
          # get the inputs; data is a list of [inputs, labels]
          inputs, labels = data
          #
          # zero the parameter gradients
          optimizer.zero_grad()
          #
          # forward + backward + optimize
          outputs = model(inputs)
          loss = criterion(outputs, labels)
          loss.backward()
          optimizer.step()
          #
          # print statistics
          running_loss += loss.item()
          if i % 2000 == 1999:    # print every 2000 mini-batches
              print('[%d, %5d] loss: %.3f' %
                    (epoch + 1, i + 1, running_loss / 2000))
              running_loss = 0.0
      #
      print('Finished Training')
    • However, I keep getting this error in the console:
      然而,控制台始终报错:

    RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same`

    • Any thoughts on how to fix it? I read that maybe the model hasn\’t been pushed into my GPU, but not sure how to fix it. Thanks!
      怎么解决?我感觉应该是模型没有送到 GPU 里,但不太清楚具体怎么解决。
  • Answers:

    • Nicolas Gervais – vote: 172

    • You get this error because your model is on the GPU, but your data is on the CPU. So, you need to send your input tensors to the GPU.
      这个报错是因为你的模型在 GPU,而数据在 CPU。因此需要把输入的张量送到 GPU。

    • inputs, labels = data                         # this is what you had
      inputs, labels = inputs.cuda(), labels.cuda() # add this line
    • Or like this, to stay consistent with the rest of your code:
      或者这样,和其它代码保持一致:

    • device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
      inputs, labels = inputs.to(device), labels.to(device)
    • The same error will be raised if your input tensors are on the GPU but your model weights aren\’t. In this case, you need to send your model weights to the GPU.
      如果输入张量在 GPU,而模型权重不是的时候,还会出现同样的错误。这时需要把模型权重送入 GPU。

    • model = MyModel()
      if torch.cuda.is_available():
        model.cuda()
    • Here is the documentation for cuda() and cpu(), its opposite.
      见文档: cuda()cpu()

    • prosti – vote: 20

    • The new API is to use .to() method.
      有个新的 API,使用的 .to() 方法。

    • The advantage is obvious and important.Your device may tomorrow be something other than cuda:
      这样做很明显更方便,说不定明天你的设备就不再是 cuda,而是下面的其它设备了:

      • cpu
      • cuda
      • mkldnn
      • opengl
      • opencl
      • ideep
      • hip
      • msnpu
      • xla
    • So try to avoid model.cuda()It is not wrong to check for the device
      所以尽量避免 model.cuda()

    • dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
    • or to hardcode it:
      或这样:

    • dev=torch.device("cuda") 
    • same as:
      或这样:

    • dev="cuda"
    • In general you can use this code:
      之后这样送入设备就行:

    • model.to(dev)
      data = data.to(dev)
    • tsveti_iko – vote: 2

    • As already mentioned in the previous answers, the issue can be that your model is trained on the GPU, but it\’s tested on the CPU. If that\’s the case then you need to port your model\’s weights and the data from the GPU to the CPU like this:
      正如前面其它回答提到的,这是由于模型训练在 GPU,测试在 CPU 导致的。因此需要将模型权重与数据从 GPU 移向 CPU:

    • device = args.device # "cuda" / "cpu"
      if "cuda" in device and not torch.cuda.is_available():
        device = "cpu"
      data = data.to(device)
      model.to(device)
    • NOTE: Here we still check if the configuration arguments are set to GPU or CPU, so that this piece of code can be used for both training (on the GPU) and testing (on the CPU).
      注:这里仍需核对 GPU/CPU 的配置参数,以便代码可以用于在 GPU 训练,在 CPU 测试。

You may also like...

发表评论

您的电子邮箱地址不会被公开。

− 3 = 1