MiniProfiler with cached pages, encoding issue about rack-mini-profiler HOT 20 CLOSED

Update on this. I made a test rails 3.2 app to try and narrow this down. It doesn't fail outright but mini profiler does add  to cached pages. Trying to figure out where that happens.

from rack-mini-profiler.

This was my own content. Force encoding on https://github.com/MiniProfiler/rack-mini-profiler/blob/master/lib/mini_profiler/profiler.rb#L419 just brought it to light.

from rack-mini-profiler.

Here's the situation that I was able to recreate in a vanilla 3.2.15 rails app.
The rails app is utf-8 and the db encoding is utf-8.

The issue is when mini profiler is reading data from a cached rack file and that file contains strings that could be of a binary encoding even though the data from the db was encoded as utf-8.

When MP is forcing the encoding here the fragment's encoding is ASCII-8BIT. If there are characters that are out of the UTF-8 scope such as an annoying apostrophe ’, the webserver chokes on it.

A fix that works but I'm not sure is ideal is to force the template to be utf-8 but then that converts the characters of course.

fragment.force_encoding("UTF-8")

That apostrophe now becomes ’

from rack-mini-profiler.

I'm experiencing this problem too. @ScotterC, thanks for diving into this, but I don't understand what you're describing

The issue is when mini profiler is reading data from a cached rack file and that file contains strings that could be of a binary encoding even though the data from the db was encoded as utf-8.

What do you mean by "could be"? Where does the data come from, the app or MP? I'm fairly certain nothing in my app would produce non-utf8 characters.

When MP is forcing the encoding here the fragment's encoding is ASCII-8BIT.

Why?

from rack-mini-profiler.

Yea. I'm sure my brain was fried when I wrote that.

I found that the data that was non UTF8 was from user input. The problem
happened to me when I was caching that page into a flat file and when rack
read it back in, the file's encoding was ASCII 8. The particular
characters I had from user input created the encoding error. Forcing UTF8
put the issue out of mind for now.

I am still confused about one aspect. I had assumed from my setup that data
from users was being tagged as UTF8 going into the database. So I'm
confused by encoding errors from data coming from the db.

On Saturday, November 9, 2013, John Bachir wrote:

I'm experiencing this problem too. @ScotterC https://github.com/ScotterC,
thanks for diving into this, but I don't understand what you're describing

The issue is when mini profiler is reading data from a cached rack file
and that file contains strings that could be of a binary encoding even
though the data from the db was encoded as utf-8.

What do you mean by "could be"? Where does the data come from, the app or
MP? I'm fairly certain nothing in my app would produce non-utf8 characters.

When MP is forcing the encoding here the fragment's encoding is ASCII-8BIT.

Why?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/18#issuecomment-28143341
.

Scott Carleton
Artsicle, Co-Founder / CTO
[email protected]
C. 914 400 5085

from rack-mini-profiler.

i had the problem in a context with no user input, other than cookies, where i doubt there was anything problematic but i could check later to be sure

I'm certain all our templates are UTF8 as well

On Nov 9, 2013, at 11:59 PM, Scott Carleton [email protected] wrote:

Yea. I'm sure my brain was fried when I wrote that.

I found that the data that was non UTF8 was from user input. The problem
happened to me when I was caching that page into a flat file and when rack
read it back in, the file's encoding was ASCII 8. The particular
characters I had from user input created the encoding error. Forcing UTF8
put the issue out of mind for now.

I am still confused about one aspect. I had assumed from my setup that data
from users was being tagged as UTF8 going into the database. So I'm
confused by encoding errors from data coming from the db.

On Saturday, November 9, 2013, John Bachir wrote:

I'm experiencing this problem too. @ScotterC https://github.com/ScotterC,
thanks for diving into this, but I don't understand what you're describing

The issue is when mini profiler is reading data from a cached rack file
and that file contains strings that could be of a binary encoding even
though the data from the db was encoded as utf-8.

What do you mean by "could be"? Where does the data come from, the app or
MP? I'm fairly certain nothing in my app would produce non-utf8 characters.

When MP is forcing the encoding here the fragment's encoding is ASCII-8BIT.

Why?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/18#issuecomment-28143341
.

Scott Carleton
Artsicle, Co-Founder / CTO
[email protected]
C. 914 400 5085
—
Reply to this email directly or view it on GitHub.

from rack-mini-profiler.

@jjb could be your cookies. this may be of help: http://stackoverflow.com/questions/5398889/error-incompatible-character-encodings-ascii-8bit-and-utf-8-due-to-8-bit-enco

from rack-mini-profiler.

okay i've tested it now -- clearing cookies does not change the behavior

from rack-mini-profiler.

+1. I am facing the same issue.

from rack-mini-profiler.

@jjb @mohitjain Here's my branch that I'm using that forces UTF-8. Not a great long term solution but works for me.

from rack-mini-profiler.

@ScotterC I rolled back to previous version of the gem 'rack-mini-profiler', '=0.1.28' and everything working fine now..

from rack-mini-profiler.

@mohitjain Good to know.

The current master branch is way ahead of the last release. Which is what @SamSaffron is using with discourse/discourse.

I wish we could compare the versions but there's no tags on the repo so we'd have to figure out which commits are at each release.

from rack-mini-profiler.

@ScotterC this might help:

git log -p lib/mini_profiler/version.rb

from rack-mini-profiler.

i can confirm that 0.1.31 does not work for me but 7bbd53b does

from rack-mini-profiler.

@ScotterC can you try release 0.9.0 ?

from rack-mini-profiler.

@SamSaffron I believe it's working. It is in my test app at least, but now I can't seem to produce the old issue so I'm not 100% sure.

from rack-mini-profiler.

I encountered "incompatible character encodings: ASCII-8BIT and UTF-8" error on some pages after adding heroku_rails_deflate gem. After looking at the rack-mini-profiler github page I saw the "Custom middleware ordering (required if using Rack::Deflate with Rails)" section, made the suggested changes, and everything worked again.

from rack-mini-profiler.

@ScotterC @jjb is this still an issue?

from rack-mini-profiler.

not sure, alas I haven't used it in years

On Jun 3, 2016, at 2:22 PM, Keenan Brock [email protected] wrote:

@ScotterC @jjb is this still an issue?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

from rack-mini-profiler.

@kbrock I don't think so. Will close

from rack-mini-profiler.