407 Commits
Author SHA1 Message Date
usa ad075f697d merge revision(s) 57816,57817: [Backport #13292]
fix UTF-32 valid_encoding?

	* enc/utf_32be.c (utf32be_mbc_enc_len): check arguments precisely.
	  [ruby-core:79966] [Bug #13292]

	* enc/utf_32le.c (utf32le_mbc_enc_len): ditto.

	* regenc.h (UNICODE_VALID_CODEPOINT_P): predicate for valid
	  Unicode codepoints.
	fix UTF-32 valid_encoding?

	* test/ruby/test_io_m17n.rb (TestIO_M17N#test_puts_widechar): do
	  not use invalid codepoint.  [ruby-core:79966] [Bug #13292]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_2@58103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-03-25 17:01:54 +00:00
nagachika 1e1ac8268a merge revision(s) 54129,54130,54132: [Backport #11844]
* enc/trans/JIS: update Unicode's notice. [Bug #11844]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_2@54334 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-03-28 16:26:34 +00:00
nagachika 782b9a8938 merge revision(s) 53202: [Backport #11843]
* enc/windows_1250.c: Should not use C++ style comments (C99 feature).
	  [Bug #11843]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_2@53909 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-02-23 23:25:27 +00:00
nagachika 80b1311383 merge revision(s) 52883,52884,53046,53198: [Backport #11838]
* enc/iso_8859_4.c: Added missing lower/upper-case character
	  pair (U+014A and U+014B, LATIN CAPITAL/SMALL LETTER ENG)

	* enc/iso_8859_13.c: Added three missing lower/upper-case
	  character pairs (from Kimihito Matsui)
	  (from Kimihito Matsui)

	* enc/windows_1252.c: separate from ISO-8859-1 to fix 0x80..0x9e
	  range.  [ruby-core:64049] [Bug #10097]

	* enc/iso_8859_2.c, enc/windows_1250.c: separate Windows-1250
	  from ISO-8859-2 to fix 0x80..0x9e range (from Kimihito Matsui)


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_2@53896 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-02-22 19:00:17 +00:00
nagachika 478cdf49d7 merge revision(s) 52016,52017,52019,52020,52021: [Backport #10735]
* enc/euc_jp.c (mbc_case_fold): check given string is valid or not,
	  and if invalid, return 1. [Bug #11486]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_2@52652 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-11-18 15:49:29 +00:00
akr ecedd3b224 Update dependencies.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48898 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-19 13:34:54 +00:00
nobu 638ba47303 load.c: defer static linked init
* enc/encinit.c.erb (Init_enc): initialize encdb and transdb
  directly.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48699 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-03 07:47:37 +00:00
nobu 673f1b3064 enc/prelude.rb: no encdb and transdb
* enc/prelude.rb: no longer need to load encdb and transdb here.
  Init_enc should load them if possible.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48698 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-03 07:47:11 +00:00
nobu 8299027f2e make_encmake.rb: fix typo
* enc/make_encmake.rb: fix typo, and use real read filename.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48621 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-27 16:21:51 +00:00
akr 6ca202fcee Update dependency.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48526 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-21 03:04:44 +00:00
akr 20fe728a24 Dependencies updated.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48483 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-18 15:28:08 +00:00
akr 4ac7afa500 * enc/encdb.c: Include internal.h.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48481 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-18 15:24:41 +00:00
akr 2b9191e557 * internal.h: Gather declarations in non-header files.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48480 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-18 15:13:05 +00:00
akr dd547e1b22 Mark auogenerated part.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48441 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-15 11:14:08 +00:00
akr 57682b05e1 * common.mk: Remove comments in Dependency lines.
Notified by usa.

* enc/depend: Ditto.

* ext/**/depend: Ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-15 10:28:25 +00:00
akr 8727eb77be * tool/update-deps: Extend to fix dependencies.
* common.mk: Dependencies updated by tool/update-deps.

* enc/depend: Ditto.

* ext/**/depend: Ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48431 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-15 05:15:49 +00:00
usa ee4782f774 * enc/prelude.rb: we sometimes run ruby without library path (especially
for test), so should permit to run ruby if unicode_normalize.rb is
  missing.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48060 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-20 15:01:13 +00:00
duerst 824670aa9e enc/prelude.rb: Added automatic loading of lib/unicode_normalize.rb. This makes sure that all
the methods that are available on String are available without explicit require.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48023 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-19 09:39:41 +00:00
naruse 106075eac8 Re-apply r43023 which is accidentally reverted by r47598
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47615 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-09-17 14:53:08 +00:00
naruse d2a5354255 * reg*.c: Merge Onigmo 5.15.0 38a870960aa7370051a3544
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-09-15 16:18:41 +00:00
nobu f587347c28 euckr-tbl.rb: euro and registered signs
* enc/trans/euckr-tbl.rb (EUCKR_TO_UCS_TBL): add missing euro and
  registered signs.  [ruby-core:64452] [Bug #10149]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47221 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-08-19 13:22:46 +00:00
naruse 64c81e40d4 * regcomp.c: Merge Onigmo 5.14.1 25a8a69fc05ae3b56a09.
this includes Support for Unicode 7.0 [Bug #9092].

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46831 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-07-16 03:27:25 +00:00
kazu 72912ec3f3 fix usage
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46317 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-02 13:46:33 +00:00
nobu 046831094b constify rb_encoding and OnigEncoding
* include/ruby/encoding.h: constify `rb_encoding` arguments.
* include/ruby/oniguruma.h: constify `OnigEncoding` arguments.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46309 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-01 22:06:11 +00:00
nobu dd20f90408 unicode.c: no initialization
* enc/unicode.c (init_case_fold_table): no longer need to
  initialize tables at runtime.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46273 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-30 23:58:34 +00:00
nobu 7e67b39679 case-folding.rb: perfect hash for case unfolding3
* enc/unicode/case-folding.rb (lookup_hash): make perfect hash to
  lookup case unfolding table 3.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46272 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-30 23:58:24 +00:00
nobu 8f59867651 case-folding.rb: perfect hash for case unfolding2
* enc/unicode/case-folding.rb (lookup_hash): make perfect hash to
  lookup case unfolding table 2.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46271 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-30 23:58:14 +00:00
nobu 35348a0806 case-folding.rb: perfect hash for case unfolding1
* enc/unicode/case-folding.rb (lookup_hash): make perfect hash to
  lookup case unfolding table 1.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46270 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-30 23:58:01 +00:00
nobu c39e659263 case-folding.rb: perfect hash for case folding
* enc/unicode/case-folding.rb (lookup_hash): make perfect hash to
  lookup case folding table.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46269 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-30 23:57:45 +00:00
nobu 88eae35862 case-folding.rb: merge tables
* enc/unicode/case-folding.rb (print_table): merge non-locale and
  locale tables, and reduce initializing loops.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46268 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-30 23:56:00 +00:00
nobu 1d588b4fde case-folding.rb: modularize
* enc/unicode/case-folding.rb (CaseFolding): modularize, and add
  --output-file option.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46267 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-30 23:55:00 +00:00
nobu 0148bd15e4 case-folding.rb: conversion script
* enc/unicode/case-folding.rb: script to convert CaseFolding.txt,
  tranlated from CaseFolding.py.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46266 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-30 23:49:54 +00:00
nobu 90fb7538f8 enc/unicode.c: lookup functions
* enc/unicode.c (onigenc_unicode_{fold,unfold{1,2,3}}_lookup):
  abstract lookup functions.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46057 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-23 04:44:15 +00:00
nobu dcaf699ee9 enc/unicode.c: constify
* enc/unicode.c (code{2,3}_{cmp,hash}): constify and adjust
  argument types.

* enc/unicode.c (onigenc_unicode_fold_lookup): constify.

* enc/unicode.c (onigenc_unicode_apply_all_case_fold): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46056 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-23 04:37:02 +00:00
nobu 7f2c371fba Makefile.in: suppress warnings
* Makefile.in (enc/jis/props.h): suppress `pointer-to-int-cast`
  warnings by using `offsetof`.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46052 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-22 15:09:11 +00:00
nobu 095e1f8eaf jis/props.kwd: constify
* enc/jis/props.kwd (PropertyList): constify table itself.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46051 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-22 14:34:58 +00:00
nobu bd403755d0 remove trailing spaces
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46044 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-22 10:58:08 +00:00
nobu f1b49b7bb9 constify character property tables
* enc/jis/props.kwd: constify character property tables of JIS
  based encodings by perfect hash.
* enc/euc_jp.c, enc/shift_jis.c: use character property functions.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46039 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-22 05:02:15 +00:00
nobu 6224216294 unused functions and variables
* complex.c, rational.c: remove unused functions, which are warned
  by clang 5.1, and also variables only used by removed functions.

* ext/date/date_core.c: ditto.

* enc/utf_16be.c, enc/utf_16le.c: comment out constants only used
  by commented out functions.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-03-17 04:57:14 +00:00
nobu d8441fcc6e encdb.c, encoding.c: make BOM-encodings dummy
* enc/encdb.c (ENC_DUMMY_UNICODE): make BOM-encodings dummy.
* encoding.c (enc_autoload): keep dummy encodings dummy.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43034 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-09-24 08:15:18 +00:00
nobu 3a2aca5075 encdb.c, utf_16_32.h: Unicode with BOM
* enc/encdb.c, enc/utf_16_32.h (ENC_DUMMY_UNICODE): Unicode with BOM
  must be based on big endian variants, so that actual encodings would
  work.  [ruby-core:57318] [Bug #8940]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43023 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-09-23 13:16:11 +00:00
naruse 0e92ae9636 * enc/trans/utf8_mac-tbl.rb: fix r42789.
Fix conversion table and logic. [ruby-dev:47680]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42823 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-09-04 06:40:39 +00:00
naruse 853c346dde * enc/trans/utf8_mac-tbl.rb: update conversion table to recent OS X.
Previous table is used on Mac OS X 10.1 or prior.
  This table is used on 10.2 or later. [ruby-dev:47680]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42789 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-09-02 23:36:01 +00:00
naruse 9ae509504b Revert r41786 "broken utf-8"
revert unexpected commit

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-07-09 07:38:15 +00:00
naruse ed37bfcc9d broken utf-8
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-07-05 00:54:11 +00:00
ktsj 166d8dc2d6 * enc/trans/japanese_euc.trans, test/ruby/test_transcode.rb,
tool/transcode-tblgen.rb: change EUC-JP-2004 to EUC-JIS-2004.
  This is follow up to changes in r41024.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41035 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-06-02 14:36:41 +00:00
naruse b285af6468 * enc/euc_jp.c: fix typo: the name of EUC-JIS-2004.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41024 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-06-01 17:11:48 +00:00
nobu a436540271 ext/depend: fix dependency for nmake
* ext/depend (ENCOBJS, TRANSOBJS): use explicit path to ruby.h for
  nmake.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-04-08 14:56:01 +00:00
nobu 1d42889c4e ext/depend: fix dependency
* ext/depend (ENCOBJS, TRANSOBJS): fix header dependency, VPATH has
  $(srcdir)/include/ruby but not $(srcdir)/include, so cannot find out
  ruby/ruby.h.  use ruby.h instead and ../ruby for include/ruby.h.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40186 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-04-08 14:09:42 +00:00
yugui 4fe59003f0 * ext/depend (ENCOBJS, TRANSOBJS): Add missing dependencies.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40185 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-04-08 11:31:38 +00:00