In tensor-house/supply-chain/supply-chain-reinforcement-learning.ipynb
the code

import ray.rllib.agents.ddpg as ddpg

gives the error

ModuleNotFoundError: No module named 'ray.rllib.agents'

After changing the line to

import ray.rllib.algorithms.ddpg as ddpg

there is still the problem that the config does not work

AttributeError: module 'ray.rllib.algorithms.ddpg' has no attribute 'DEFAULT_CONFIG'

I use
ray version 2.4.0
Python 3.10.9
Ubuntu

I would like to try the Next Best Action model for a seller recommendation.
Can you please list steps I need to follow to try the Next Best Action model with your dataset?

Hi Ilya,

First, thanks so much for putting together this really helpful repo!

I've been trying to understand channel-attribution-lstm, and I've got a couple of questions about your features_for_lstm function.

1. This code here:
   f_proj = df[['jid', 'campaigns', 'cats', 'click', 'cost', 'time_since_last_click_norm', \ 'timestamp_norm', 'conversion']]
x2d = df_proj.values
x3d_list = np.split(x2d[:, 1:], np.cumsum(np.unique(x2d[:, 0], return_counts=True)[1])[:-1])

Won't this split only work as intended if f_proj is sorted out by jid first? If the goal here is to separate out the sub-arrays for individual jids? Or am I missing the point?

3. journey_matrix = journey_matrix[ journey_matrix[:, 5].argsort() ] # sort impressions by timestamp

Why 5 here? The timestamp_norm field is next to last in the journey_matrix array, so shouldn't it be journey_matrix.shape[1]-2?

4. Finally, I had to convert y_train, y_val, and y_test into arrays for the model to run; it wouln't work for me as written because they were lists.

   I'd really appreciate your help! Thank you again --
   Natalia

code in promotions (https://github.com/ikatsov/tensor-house/blob/master/promotions/channel-attribution-lstm.ipynb)

'hist_all' is not defined

In [198]:

Data exploration

def journey_lenght_histogram(df):
    counts = df.groupby(['jid'])['uid'].count().reset_index(name="count").groupby(['count']).count()
    return counts.index, counts.values / df.shape[0]

hist_x, hist_y = journey_lenght_histogram(df4)

plt.plot(range(len(hist_all)), hist_all, label='all journeys')
plt.yscale('log')
plt.xlim(0, 120)
plt.xlabel('Journey length (number of touchpoints)')
plt.ylabel('Fraction of journeys')
plt.show()

@ikatsov (want to know where hist_all is defined, it's not used in any block of code before)

SimpleControlPolicy._find_source for ToyFactoryCell_3 is giving error in consumer as it is declared as None but is trying to access sum(facility.consumer.open_orders.values(), Counter()).

Please provide resolution.

Hello,

I was trying to replicate your RL approach in another but similar use case where the requirement is to offer multiple offerings at same time such as in your context both small and big discount. Please suggest how to handle that with your approach what would be the F array as we would have multiple values in same time step.

Thanks
San

Hi Ilya,

I have been experimenting with your nicely implemented module to understand 'Multi-agent deep reinforcement learning' and combinatorial optimization for 'multi-echelon supply chain'.

I hereby raise the issue since the latest RLLib module is not compatible with the code that you have created. It would be highly helpful for every one if you can provide the required modifications to make it compatible with the latest RLLib & ray package as multiple things have change in there.

Thanks
Ankit

Filename : pricing/price-optimization-using-dqn-reinforcement-learning.ipynb

at t=0, the Below function evaluates with p[0] and p[-1] as parameters which seems incorrect to me.
because p[-1] in python corresponds to last element of the array.

def profit_total(p, unit_cost, q_0, k, a, b):
  return profit_t(p[0], p[0], q_0, k, 0, 0, unit_cost) + sum(map(lambda t: profit_t(p[t], p[t-1], q_0, k, a, b, unit_cost), range(len(p))))

to fix this, we can use range(1,len(p)).

def profit_total(p, unit_cost, q_0, k, a, b):
  return profit_t(p[0], p[0], q_0, k, 0, 0, unit_cost) + sum(map(lambda t: profit_t(p[t], p[t-1], q_0, k, a, b, unit_cost), range(1,len(p))))

@ikatsov Do let me know if I am wrong or misunderstood something.

Given that the Pearson correlation coefficient is symmetric: corr(X,Y) = corr(Y,X),
I don't understand how user_similarity in your notebook matrix is asymmetry.

#image-artistic-style-similarity

After # compute styles when calling "style" facing an issue:

InvalidArgumentError: Must provide as many biases as the last dimension of the input tensor: [3] vs. [1,400,400,4] [Op:BiasAdd] name: style_model_7/BiasAdd/.

I found a reason initially if I run this:

compute styles

image_style_embeddings = {}
for image_path in tqdm(image_paths):
image_tensor = load_image(image_path)
style = style_to_vec(image_to_style(image_tensor))
image_style_embeddings[ntpath.basename(image_path)] = style

Then for some reason, the shape remains [1,400,400,4] when the tensor is empty but after when tensor start to fill the shape changes to [1,400,400,3]

But if I run this:

compute styles

image_style_embeddings = {}
for image_path in tqdm(image_paths):
->image_tensor = load_image(image_path)
->if (image_tensor.shape[3]!= 4):
-> ->style = style_to_vec(image_to_style(image_tensor))
->->image_style_embeddings[ntpath.basename(image_path)] = style

Its working fine is it ok to do?

Thanks in advance!

Hi, could you please add the Python packages as a requirements.txt file to your repository?

Fantastic job on the book and notebooks in this repository.

I was working with book-enterprise-ai-edition-2/recipe-10/dynamic-pricing-limited-experimentation.ipynb and saw one minor issue.
I wanted to see what the timesteps looked like if I used a larger value of m (Using 6 price steps max rater than 4).
When I changed m to a higher value, then eventually the logx function attempts to calculate log of a value < 1 which returns a negative number.

This is the current function:
def logx(x, n):
for i in range(0, n):
x = math.log(x) if x>0 else 0
return x

To avoid that, I beleive this line:
x = math.log(x) if x>0 else 0
should be changed to:
x = math.log(x) if x>1 else 0

From section 3.2 (Notations) here: https://dspace.mit.edu/bitstream/handle/1721.1/119156/Pricing_v4_a.pdf?sequence=1&isAllowed=y

"We use log(m)T to represent m iterations of the logarithm, log(log(... log(T))), where m is the number of price changes.
For convenience, we let log(x) = 0 for all 0 ≤ x < 1, so the value of log(m) T is defined for all T ≥ 1."

Which version of ray is required for World of Supply? Can you provide a requirements.txt?
I got this issue when trying to run WoS.

And my system information:
python: 3.7.9
ray: 1.2.0.dev0

Hi,

The above doesn't load. Please advise.

Thanks