Add supporting files for test_tokenize

Supporting files not associated with another test:
 - tokenize_tests-latin1-coding-cookie-and-utf8-bom-sig.txt
 - tokenize_tests-utf8-coding-cookie-and-no-utf8-bom-sig.txt
 - tokenize_tests-utf8-coding-cookie-and-utf8-bom-sig.txt
This commit is contained in:
Padraic Fanning committed 2021-05-26 21:04:55 -04:00
1 parent 67420d47fd
commit a12bd0fbed
3 files changed
+38

No files matched your search

@@ -0,0 +1,13 @@
# -*- coding: latin1 -*-
# IMPORTANT: this file has the utf-8 BOM signature '\xef\xbb\xbf'
# at the start of it. Make sure this is preserved if any changes
# are made! Also note that the coding cookie above conflicts with
# the presence of a utf-8 BOM signature -- this is intended.
# Arbitrary encoded utf-8 text (stolen from test_doctest2.py).
x = 'ЉЊЈЁЂ'
def y():
"""
And again in a comment. ЉЊЈЁЂ
"""
pass
@@ -0,0 +1,13 @@
# -*- coding: utf-8 -*-
# IMPORTANT: unlike the other test_tokenize-*.txt files, this file
# does NOT have the utf-8 BOM signature '\xef\xbb\xbf' at the start
# of it. Make sure this is not added inadvertently by your editor
# if any changes are made to this file!
# Arbitrary encoded utf-8 text (stolen from test_doctest2.py).
x = 'ЉЊЈЁЂ'
def y():
"""
And again in a comment. ЉЊЈЁЂ
"""
pass
@@ -0,0 +1,12 @@
# -*- coding: utf-8 -*-
# IMPORTANT: this file has the utf-8 BOM signature '\xef\xbb\xbf'
# at the start of it. Make sure this is preserved if any changes
# are made!
# Arbitrary encoded utf-8 text (stolen from test_doctest2.py).
x = 'ЉЊЈЁЂ'
def y():
"""
And again in a comment. ЉЊЈЁЂ
"""
pass