[BUG] DiscreteDecisionTransformer Inference Problem, AttributeError: 'numpy.ndarray' object has no attribute 'length' about d3rlpy HOT 4 CLOSED

@waltersharpWEI Hi, thanks for the issue. The short answer is that Decision Transformer algorithms don't support any of evaluators because they're essentially different from Q-learning-based algorithms. Please let me close this issue since it's not a bug.

from d3rlpy.

Sorry, looking at your code again, you're not using evaluators technically. For inference, please check this documentation for the usage.
https://d3rlpy.readthedocs.io/en/v2.5.0/references/algos.html#decision-transformer

from d3rlpy.

# start training (save logs to LOGS_DIR)
dt.fit(
    dataset,
    n_steps=100,
    n_steps_per_epoch=10,
    eval_target_return=0,
    # manually specify action-sampler
    eval_action_sampler=d3rlpy.algos.IdentityTransformerActionSampler(),
)
actor = dt.as_stateful_wrapper(
    target_return=0,
    action_sampler=d3rlpy.algos.IdentityTransformerActionSampler(),
)

# interaction
observation, reward = env.reset(), 0.0
for i in range(100):
    action = actor.predict(observation, reward)
    observation, reward, done, truncated, _ = env.step(action)
    if done or truncated:
        break

print("Complete../")

Thanks for your reply, I modified the code in the above form.
I just have one question. Does the inference must be done using the .as_stateful_wrapper() function?
So does that mean the predict() is not supposed to be used in the inference?

from d3rlpy.

It's kind of right. This is because of the nature of stateful behavior of Decision Transformer, which requires carefully crafted inputs. I don't think users want to see this level of complication:

from d3rlpy.