enough-software / enough_convert Goto Github PK

View Code? Open in Web Editor NEW

8.0 8.0 9.0 690 KB

Support for ISO 8859 / Latin and Windows character encodings missing from `dart:convert`.

License: Mozilla Public License 2.0

Dart 100.00%

character character-encodings charset dart decoder encoder encoding flutter

enough_convert's People

Contributors

Stargazers

Watchers

Forkers

habbas11 ohayak pavels0 rascal-3 supermail-one in4matic-srl rk0cc diskominfopematangsiantar nidal-bakir

enough_convert's Issues

FormatException: Invalid value in input: \"➢\" / (10146) at index 1204

i am getting this error when copying and pasting text from word into an editable div and trying to write to the database by encoding the data with the enough_convert win1252 codec

it seems the codec doesn't know how to deal with the "➢" character

FormatException: Invalid value in input: \"➢\" / (10146) at index 1204 of \""

#0      BaseEncoder.convert (package:enough_convert/src/base.dart:166:13)
#1      Codec.encode (dart:convert/codec.dart:21:32)
#2      new ParameterValue.text (package:postgres_fork/src/query.dart:313:20)
#3      new ParameterValue (package:postgres_fork/src/query.dart:296:29)
#4      Query._formatSql.<anonymous closure> (package:postgres_fork/src/query.dart:119:20)
#5      Query._prepareSubstitutionValues (package:postgres_fork/src/query.dart:83:13)
#6      Query._formatSql (package:postgres_fork/src/query.dart:115:9)
#7      Query.sendExtended (package:postgres_fork/src/query.dart:149:23)
#8      _PostgreSQLConnectionStateReadyInTransaction.processQuery (package:postgres_fork/src/connection_fsm.dart:376:9)
#9      _PostgreSQLConnectionStateReadyInTransaction.awake (package:postgres_fork/src/connection_fsm.dart:362:14)
#10     _PostgreSQLConnectionStateReadyInTransaction.onEnter (package:postgres_fork/src/connection_fsm.dart:355:12)
#11     PostgreSQLConnection._transitionToState (package:postgres_fork/src/connection.dart:302:41)
#12     PostgreSQLConnection._readData (package:postgres_fork/src/connection.dart:337:11)
#13     _rootRunUnary (dart:async/zone.dart:1399:47)
#14     _CustomZone.runUnary (dart:async/zone.dart:1300:19)
#15     _CustomZone.runUnaryGuarded (dart:async/zone.dart:1209:7)
#16     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#17     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#18     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:774:19)
#19     _StreamController._add (dart:async/stream_controller.dart:648:7)
#20     _StreamController.add (dart:async/stream_controller.dart:596:5)
#21     _Socket._onData (dart:io-patch/socket_patch.dart:2324:41)
#22     _rootRunUnary (dart:async/zone.dart:1407:13)
#23     _CustomZone.runUnary (dart:async/zone.dart:1300:19)
#24     _CustomZone.runUnaryGuarded (dart:async/zone.dart:1209:7)
#25     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#26     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#27     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:774:19)
#28     _StreamController._add (dart:async/stream_controller.dart:648:7)
#29     _StreamController.add (dart:async/stream_controller.dart:596:5)
#30     new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1849:33)
#31     _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:1322:14)
#32     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
#33     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
#34     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:122:13)
#35     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:193:5)

Support windows 1254 encoding

It should support windows 1254 encoding

Support Big5 character encoding?

Big5 is used in Taiwan, compare
https://en.wikipedia.org/wiki/Big5

Support chunked conversion

Currently no chunked conversion is supported, it would be nice to suport decoder/encoder .startChunkedConversion(...).

How to convert from Windows1251 String to UTF String?

This is more of a question rather than an issue, but hopefully someone can answer me.

I am loading the HTML from a Russian webpage using http package into a String variable like so:

String html = await http.Client().get(Uri.parse(url)).body;

The website encoding is Windows1251. So for example html variable can have text such as "Êàêèå". This is what I see when I print the variable.

So my question is: How do I convert that string to Cyrillic characters in Unicode encoding which should result in "Какие"?

I tried this:

import 'dart:convert';
import 'package:enough_convert/enough_convert.dart';

void main() {
  final html = "Êàêèå";
  final encoded = const Windows1251Codec().encode(html);
  final converted = const Utf8Codec().decode(encoded);
  print(converted);
}

But I get an error on this line: final encoded = const Windows1251Codec().encode(html);:

FormatException: Invalid value in input: "Ê" / (202) at index 0 of "Êàêèå"

Essentially what I would like to do is to convert "Êàêèå" to "Какие". You can do this on the website https://convertcyrillic.com. Here is the screenshot:

So how do I do this programmatically in Dart?

how to use this with 'http' or 'dio' package to send formdata?

hello there ~
I want send datas using gbk encoding
but http package not support gbk

var datas = {
   "message" : "你好"
}
final res = await http.post(_uri, body: datas, encoding: GbkCodec());

OUTPUT:
Unhandled Exception: FormatException: Unsupported encoding "gbk".

Please add Mazovia support

Could you please add support for Mazovia encoding? https://en.wikipedia.org/wiki/Mazovia_encoding
It is used in thermal printers.
Thank you in advance

add StreamTransformer suport

import 'dart:convert';
import 'dart:io';

import 'package:csv/csv.dart';
import 'package:enough_convert/enough_convert.dart';

void main(List<String> args) async {
  final input = new File(
          'C:/MyDartProjects/notifis/notifis_backend/db/planilhas/CIP 2012.csv')
      .openRead();

  final codec = const Windows1252Codec(allowInvalid: false);
  final stream =  await input.transform(codec).transform(LineSplitter());
  await for (var lineString in stream) {
    var rowsAsListOfValues =
        CsvToListConverter(fieldDelimiter: ';').convert(lineString);
    var lineCsv = rowsAsListOfValues.first;
    print(lineCsv.map((e) => '$e').join(' | '));
  }

  exit(0);
}

pub outdated --mode=null-safety
dart pub upgrade --null-safety
dart migrate

add function iconv like PHP

//$string = '➢ Isaque';
 var_dump(iconv('UTF-8','ISO-8859-1//IGNORE',pack('C*', ...[226, 158, 162, 73, 115, 97, 113, 117, 101])));  
//string(6) "Isaque"

https://www.php.net/manual/pt_BR/function.iconv.php
https://github.com/ashtuchkin/iconv-lite

please add cp-850 support

for decoding text in excel MS DOS CSV sheets

Support gb2312 encoding

GB/T 2312-1980 is a key official character set of the People's Republic of China, used for Simplified Chinese characters. GB2312 is the registered internet name for EUC-CN.

https://en.wikipedia.org/wiki/GB_2312