Problem:

The csv file quite big ~25MB, and a quite difficult to process in Excel/Numbers/Spreadsheet software.
As a programmer, I want to get the first 1000 rows using Python.

Solution:

import pandas as pd

df = pd.read_csv("DATA_CUSTOM.csv")

print(df.head(2000).to_csv("export-1k-rows.csv"))

Lesson:

• Find a smart way to process things.
• Take advantage of programming language and library, for example in this case are: python and pandas.

Problem:

translation data {:zero=>"%A, %B %{day_ordinalize}, %Y\n%A, %B %{day_ordinalize},

Solution:

https://github.com/ruby-i18n/i18n/blob/e4ce5e58f0524ae7c34ca94971363e13aa889f36/test/fixtures/locales/plurals.rb
https://makandracards.com/makandra/47056-dealing-with-i18n-invalidpluralizationdata-errors

Lesson:

• Review the error message over again and again until understand each char - the message is pretty clear.
• Dig into the log, check for param to understand the context
• Dig into the gem code

Problem:

https://yehudakatz.com/2010/02/01/safebuffers-and-rails-3-0/

Solution:

Lesson:

Problem:

As a developer, I want to set up https for my local development, so I can access https://localhost:3000 to test the features which require https.

Solution:

Generate SSL certificate: https://github.com/codica2/rails-puma-ssl

Upgrade puma gem to version 4.0

• puma/puma#1670 (comment)

Start the development server:

rails s -b 'ssl://lvh.me:3000?key=localhost.key&cert=localhost.crt'

Then access the website:

https://localhost:3000

Lesson:

• On the local http, the js loading will be blocked. Check if the current js load by https
• https://medium.com/@dblazeski/chrome-bypass-net-err-cert-invalid-for-development-daefae43eb12

Ruby PORO quite confuse me sometimes, let talk a look at this example:

class TardisMotoForm
  # Include the ActiveModel
  include ActiveModel::Model
  include ActiveModel::Validations::Callbacks

  # Define a custom constant
  PASSWORD_FORMAT = /(?=.*\d)(?=.*[a-z])(?=.*[A-Z])/

  # Define attr_accessor, so we can expose these attributes to the outside world
  # take a look more on attr_reader and attr_accessor
  attr_accessor :user, :email, :password, :phone_number, :transfer_type, :params, :taken_email, :params

  # Use the validation of model
  validate :email_uniqueness
  validates :password, presence: true, format: { with: PASSWORD_FORMAT }, length: { in: Devise.password_length }, if: :transfer_by_email?
  validates :email, email: { validate_mx: false }
  phony_normalize :phone_number
  validates :phone_number, phony_plausible: true

  # Define a custom method for the outside world can invoke
  # For example, in the controller
  # if @tardis_moto_form.valid_params?(filtered_fun_params)
  #   redirect_to somewhere_path
  # else
  #   render :new
  # end
  def valid_params?(params)
    assign_attributes(filter_params(params)) # where the assign_attribute come from?
    valid? 
  end

  # In the controller
  # if @tardis_moto_form.save(filtered_fun_params, resource) && resource.persisted?
  # end
  def save(attributes = {}, user = nil)
    @user = user # What is the user here? - it will assign to the attr user above
    @params = attributes

    user.errors.empty?
  end

  private

  # Model validation callback
  def transfer_by_email?
    transfer_type.to_sym == :email
  end

  # Model validation call back
  def email_uniqueness
    return unless User.exists?(email: email)

    @taken_email = true
    errors.add(:email, I18n.t('activerecord.errors.models.transfer_form.taken_email'))
  end

  # Sanitize filter params before assign attribute 
  def filter_params(params)
    {
      email: params[:email],
      password: params[:password],
      phone_number: params[:phone_number],
    }
  end
end

Reference:

• https://guides.rubyonrails.org/active_record_validations.html#valid-questionmark-and-invalid-questionmark
• https://api.rubyonrails.org/classes/ActiveModel/AttributeAssignment.html

Problem:

Solution:

Find the elements which have attributes in common.
Add a class to manage it easier.

Lesson:

Problem:

class TwoFer
  def two_fer
    puts "1"
  end
end

When I tried to call TwoFer.two_fer then it will show the error:

Error:
TwoFerTest#test_no_name_given:
NoMethodError: undefined method `two_fer' for TwoFer:Class
    two_fer_test.rb:8:in `test_no_name_given'

Solution:

Lesson:

• Classes and instances (and class methods and instance methods)

Problem:

Source: Ruby Performance Optimization

What’s the deal with the Ruby GC?
Did our code use too much memory?
Is the Ruby GC too slow?
The answer is a resounding yes to both questions.

High memory consumption is intrinsic to Ruby. It’s a side effect of the language design. “Everything is an object” means that programs need extra memory to represent data as Ruby objects. Also, slow garbage collection is a well-known historical problem with Ruby. Its mark-and-sweep, stop-the-world GC is not only the slowest known garbage collection algorithm. It also has to stop the application for the time GC is running. That’s why our application takes almost a dozen seconds to complete.

List of memory-inefficient operations

• literal syntaxes

"this is string literal".length

Source: The Ruby Programming Language
Object Lifetime (3.8.2)

An object becomes a candidate for garbage collection when it is unreachable—when there are no remaining references to the object except other unreachable objects.

The fact that Ruby uses garbage collection means that Ruby programs are less susceptible to memory leaks than programs written in languages that require objects and memory to be explicitly deallocated and freed. But garbage collection does not mean that memory leaks are impossible: any code that creates long-lived references to objects that would otherwise be short-lived can be a source of memory leaks.

Consider a hash used as a cache. If the cache is not pruned using some kind of least-recently-used algorithm, then cached objects will remain reachable as long as the hash itself is reachable. If the hash is referenced through a global variable, then it will be reachable as long as the Ruby interpreter is running.

Lessons:

The 80-20 rule of Ruby performance optimization: 80% of performance improvements come from memory optimization, so optimize memory first.

References:

What is GC in Ruby?
https://ruby-doc.org/core-2.4.0/GC.html

Understand the GC Stat
https://www.speedshop.co/2017/03/09/a-guide-to-gc-stat.html

https://www.toptal.com/ruby/hunting-ruby-memory-issues
https://brandur.org/ruby-memory
https://www.monitis.com/blog/20-ruby-performance-tips
https://blog.peterzhu.ca/notes-on-ruby-gc

Problem:

When use Devise with Devise security gem for ensuring the password is always strong enough.

If the user login via OAuth without password, we have to generate a random password for this user in order to bypass the NOT NULL of the encrypted_password in users table as default.

password = SecureRandom.urlsafe_base64

or

password = Devise.friendly_token[0,20]

Sometimes the result of urlsafe_base64 returns a string without number, then it causes password insecurity according to the security extension devise_security_extension

You can check by comparing the generated value of urlsafe_base64 to the pattern of strong password in the security gem config

[16] pry(main)> password_regex.match?(SecureRandom.urlsafe_base64)
=> true
[17] pry(main)> password_regex.match?(SecureRandom.urlsafe_base64)
=> true
[18] pry(main)> password_regex.match?(SecureRandom.urlsafe_base64)
=> true
[19] pry(main)> password_regex.match?(SecureRandom.urlsafe_base64)
=> false
[20] pry(main)> password_regex.match?(SecureRandom.urlsafe_base64)
=> true

The Devise has provided a method but still uses the same urlsafe_base64 method, therefore this case is still possible to happen.

  # Generate a friendly string randomly to be used as token.
  # By default, the length is 20 characters.
  def self.friendly_token(length = 20)
    # To calculate real characters, we must perform this operation.
    # See SecureRandom.urlsafe_base64
    rlength = (length * 3) / 4
    SecureRandom.urlsafe_base64(rlength).tr('lIO0', 'sxyz')
  end

Solution:

Override the default generator of Devise password.

Lesson:

1. Dig into the gem source to see how the data is generated.

2. By the urlsafe_base64 sometimes return a string without number?

Reference:

https://stackoverflow.com/questions/3681827/how-to-auto-generate-passwords-in-rails-devise

Problem:

As developer, I want to test the model validation rules and relationship

Solution:

Lesson:

Problem:

Solution:

https://stackoverflow.com/questions/38948102/center-and-right-align-flexbox-elements

Lesson:

What is flex: 1?
What is flex-grow: 1?

Problem:

master > develop > feature_a > feature_b > feature_c

feature_b branch is based on feature_a, and feature_c is based on feature_b.

It sounds a bit strange but in real-life case, there are some situations like that.

If the feature_a has 20 commits, then when feature_b rebase, it has to go through 20 commits :( quite nightmare.

Solution:

Squash all commit in feature_a to 1 commit.
But if the feature_b branch before feature_a squash, please rebase again make sure the commit is consistent between 2 branches.

Lesson:

• https://blog.carbonfive.com/always-squash-and-rebase-your-git-commits/

Problem:

Today I work with a model, which is defined default scope.
But i need to get the data which including the inactive status.

module SuperStore
  class StoreItem < ApplicationRecord
    # Enums
    enum status: [:inactive, :active], _suffix: true

    # Scopes
    default_scope { active_status.order(:type, updated_at: :desc) }
  end
end

Solution:

Override by adding this.

scope :with_inactive_status, -> { unscope(where: :status) }

Then use it as:

def set_resource
  @store_item = StoreItem.with_inactive_status.find(params[:id])
end

Lesson:

• Is the default scope good?
  https://coderwall.com/p/khht6a/beware-of-using-default-scope

• We should not use unscoped, define a scope in model to make it clearer
  https://stackoverflow.com/questions/25087336/why-is-using-the-rails-default-scope-often-recommend-against
  https://stackoverflow.com/questions/9968738/how-to-bypass-default-scope
  https://stackoverflow.com/questions/1834159/overriding-a-rails-default-scope/4166950#4166950
  https://qiita.com/yokochi@github/items/730418b0c7f36ded5b5f

Problem:

When release new features, however the customer want hide it for some purpose:

• Testing
• For specific user
• Event days

Solution:

Define variable in env.
Use that flag to control the logic.

NEW_REGISTRATION_ENABLED: false
AWESOME_FEATURE_ENABLED: false

Lesson:

https://jeroenmols.com/blog/2019/08/13/featureflags

Problem:

class ThirdPartySerializer < ActiveModel::Serializer
  attributes :session_id, :token, :started_at

  def token
    additional_data = {
      role: current_user.role,
      userId: current_user.id
    }.to_json

    THIRD_PARTY.generate_token(object.session_id, data: additional_data)
  end
end

In this context, the current_user is undefined.

Solution:

Add a private get current user to the serializer.

class ThirdPartySerializer < ActiveModel::Serializer
  #....(existing code)

  private

  def current_user_fallback
    @current_user_fallback ||= @instance_options[:current_user] || current_user
  end
end

Lesson:

• https://stackoverflow.com/questions/47809626/how-to-use-devise-current-user-in-active-model-serializers
• rails-api/active_model_serializers#624 (comment)
• https://github.com/rails-api/active_model_serializers/blob/0-10-stable/docs/general/serializers.md#controller-authorization-context

Problem:

I don't understand this code for set default function in Elixir

defmodule Sample.Default.Params do
  def first(list, val \\ nil)
  def first([head | _], _), do: head
  def first([], val), do: val
end

# defmodule Sample.Default.Params do
#   def first(list, val \\ nil), do: val # Why we remove this one?
#   def first([head | _], _), do: head
#   def first([], val), do: val
# end

# The warnning message
# warning: this clause for first/2 cannot match because a previous clause at line 2 always matches
#   sample_default_params.exs:3
# warning: this clause for first/2 cannot match because a previous clause at line 2 always matches
#   sample_default_params.exs:4

# Test cases
# Sample.Enum.first([1,2,3]) # => 1
# Sample.Enum.first([], 1) # => 1
# Sample.Enum.first([]) # => nil
# Sample.Enum.first([1,2,3], 0) # => 1

Solution:

Lesson:

Problem:

Bit confuse about String Literal in Ruby.

Solution:

What is String?
Object in Ruby.

What is Literal?
The value appears directly in Ruby code.

Example:

╰─$ irb
2.4.0 :005 > 5.times { a = 'test'; puts a.object_id; }
70105430666100
70105430665960
70105430665840
70105430665760
70105430665700
 => 5
2.4.0 :006 > 5.times { a = 'test'.freeze; puts a.object_id }
70105426108320
70105426108320
70105426108320
70105426108320
70105426108320
 => 5

Lesson:

According to The Ruby Programming Language book:

For efficiency, you should avoid using literals within loops.

Problem:

As an user i want to login via my Facebook account.
And the website app can retrieve my Facebook info (email, public info) to create new account in system.

Solution:

Lesson:

1. Use Ngrok to enable public access to local development webserver.
2. Provide the term and privacy page to Facebook App configuration to make it live.
3. Check correct callback URL, it often includes the https
4. Allow Rails app access from Ngrok, by adding the below code to file:

config/environments/development.rb

config.hosts << "a2395ad5fbe8.ngrok.io"

5. Config Devise

config/initializers/devise.rb

config.omniauth :facebook, APP_CONFIG.facebook.app_id, APP_CONFIG.facebook.secret, scope: APP_CONFIG.facebook.scope

File: config/settings/application.rb

  group :facebook do
    set :namespace, 'FACEBOOK_NAMESPACE'
    set :app_id, 'FACEBOOK_APP_ID'
    set :secret, 'FACEBOOK_SECRET'
    set :scope, 'email, public_profile, user_birthday, user_likes'
    set :cache_expiry_time, 7.days
  end

Problem:

Normally Macbook or Ubuntu will be default with the python 2.7

─$ python2 -V
Python 2.7.16

How to install new library for python3.

python3 -m pip install pandas

Solution:

• https://opensource.com/article/19/5/python-3-default-mac

Lesson:

Find the most elegant way to change the default version of Python.