Django/モデル検索時の処理を読む（all）
をテンプレートにして作成
[
トップ
] [
新規
|
一覧
|
単語検索
|
最終更新
|
ヘルプ
]
開始行:
[[Djangoを読む]]
#contents
*はじめに [#w3526026]
「APIで遊んでみる」の冒頭、saveメソッドが最終的にINSERTに...
その後、値を変更してもう一度saveメソッドを呼び出していま...
とすると次の読解対象は、
# objects.all() displays all the questions in the databa...
>>> Question.objects.all()
<QuerySet [<Question: Question object>]>
です。
*django/db/models/query.py [#dcb7ac74]
**QuerySet [#xef445b6]
objectsは前回も少し触れましたがManagerクラスのインスタン...
#code(Python){{
def all(self):
"""
Returns a new QuerySet that is a copy of the curr...
QuerySet to proxy for a model manager in some cas...
"""
return self._clone()
}}
シンプル。allメソッドを呼んだ時点ではQuerySetのコピーが作...
***__repr__ [#r5e3fa62]
シェルを使う場合、repr関数を使ってオブジェクトの表示が行...
#code(Python){{
def __repr__(self):
data = list(self[:REPR_OUTPUT_SIZE + 1])
if len(data) > REPR_OUTPUT_SIZE:
data[-1] = "...(remaining elements truncated)...
return '<QuerySet %r>' % data
}}
***__getitem__ [#i05b761a]
__getitem__に進む。初めに書いてある不正値チェックは省略。
#code(Python){{
def __getitem__(self, k):
"""
Retrieves an item or slice from the set of results.
"""
if self._result_cache is not None:
return self._result_cache[k]
if isinstance(k, slice):
qs = self._clone()
if k.start is not None:
start = int(k.start)
else:
start = None
if k.stop is not None:
stop = int(k.stop)
else:
stop = None
qs.query.set_limits(start, stop)
return list(qs)[::k.step] if k.step else qs
qs = self._clone()
qs.query.set_limits(k, k + 1)
return list(qs)[0]
}}
kはsliceです。で、
:start|None
:stop|REPR_OUTPUT_SIZE + 1 = 21
:step|None
です。
一応、set_limitsを見ておきましょう。queryはsqlモジュール...
#code(Python){{
def set_limits(self, low=None, high=None):
"""
Adjusts the limits on the rows retrieved. We use ...
as it makes it more Pythonic to read and write. W...
created, they are converted to the appropriate of...
Any limits passed in here are applied relative to...
constraints. So low is added to the current low v...
clamped to any existing high value.
"""
if high is not None:
if self.high_mark is not None:
self.high_mark = min(self.high_mark, self...
else:
self.high_mark = self.low_mark + high
if low is not None:
if self.high_mark is not None:
self.low_mark = min(self.high_mark, self....
else:
self.low_mark = self.low_mark + low
if self.low_mark == self.high_mark:
self.set_empty()
}}
無駄にややこしい(笑)。high_markの初期値はNone、low_markの...
:high_mark|21
:low_mark|0
という値が設定されます。
***__iter__ [#fbf8c40b]
さて、処理はQuerySetに返ってきます。stepは設定されてない...
#code(Python){{
def __repr__(self):
data = list(self[:REPR_OUTPUT_SIZE + 1])
if len(data) > REPR_OUTPUT_SIZE:
data[-1] = "...(remaining elements truncated)...
return '<QuerySet %r>' % data
}}
listはiterableを受け付けるので、__iter__が呼ばれるはずで...
#code(Python){{
def __iter__(self):
"""
The queryset iterator protocol uses three nested ...
default case:
1. sql.compiler:execute_sql()
- Returns 100 rows at time (constants.GET_...
using cursor.fetchmany(). This part is r...
doing some column masking, and returning...
2. sql/compiler.results_iter()
- Returns one row at time. At this point t...
tuples. In some cases the return values ...
Python values at this location.
3. self.iterator()
- Responsible for turning the rows into mo...
"""
self._fetch_all()
return iter(self._result_cache)
}}
コメントを見るとややこしそうですが、コードを見る限り、レ...
***_fetch_all [#o43dd085]
_fetch_allに進みます。
#code(Python){{
def _fetch_all(self):
if self._result_cache is None:
self._result_cache = list(self.iterator())
if self._prefetch_related_lookups and not self._p...
self._prefetch_related_objects()
}}
prefetchは無視。iteratorメソッドに進みます。
#code(Python){{
def iterator(self):
"""
An iterator over the results from applying this Q...
database.
"""
return iter(self._iterable_class(self))
}}
尻尾がつかめたと思ったらまた離れた感があります。_iterable...
**ModelIterable [#sb8a9d38]
ModelIterableクラスはdjango/db/models/query.pyの上の方に...
#code(Python){{
def __iter__(self):
queryset = self.queryset
db = queryset.db
compiler = queryset.query.get_compiler(using=db)
# Execute the query. This will also fill compiler...
# and annotations.
results = compiler.execute_sql()
select, klass_info, annotation_col_map = (compile...
compile...
model_cls = klass_info['model']
select_fields = klass_info['select_fields']
model_fields_start, model_fields_end = select_fie...
init_list = [f[0].target.attname
for f in select[model_fields_start:m...
related_populators = get_related_populators(klass...
for row in compiler.results_iter(results):
obj = model_cls.from_db(db, init_list, row[mo...
if related_populators:
for rel_populator in related_populators:
rel_populator.populate(row, obj)
if annotation_col_map:
for attr_name, col_pos in annotation_col_...
setattr(obj, attr_name, row[col_pos])
# Add the known related objects to the model,...
# 省略
yield obj
}}
+まずcompiler取得してSQL実行
+結果それぞれに対してモデルを取得、yieldで情報を渡す
ということが行われています。ぱっと見な感じだとrelated_pop...
*django.db.models.sql.compiler.SQLCompiler [#h81a1c13]
今回はqueryはQueryオブジェクトなので生成されるコンパイラ...
execute_sqlメソッドを見てみましょう。長いのでエラー処理と...
#code(Python){{
def execute_sql(self, result_type=MULTI):
"""
Run the query against the database and returns th...
return value is a single data item if result_type...
iterator over the results if the result_type is M...
"""
try:
sql, params = self.as_sql()
except EmptyResultSet:
# 省略
cursor = self.connection.cursor()
try:
cursor.execute(sql, params)
except Exception:
# 省略
# result_typeがMULTI以外の処理
result = cursor_iter(
cursor, self.connection.features.empty_fetchm...
self.col_count
)
if not self.connection.features.can_use_chunked_r...
try:
# If we are using non-chunked reads, we r...
# structure as normally, but ensure it is...
# before going any further.
return list(result)
finally:
# done with the cursor
cursor.close()
return result
}}
やっていることは、
+SQLを構築して
+実行して
+結果をイテレータとして返す
ということになります。ちなみに、sqlite3の場合、can_use_ch...
**as_sql [#ia7abe09]
as_sqlも長いので必要ないところは省略すると、
#code(Python){{
def as_sql(self, with_limits=True, with_col_aliases=F...
"""
Creates the SQL for this query. Returns the SQL s...
parameters.
If 'with_limits' is False, any limit/offset infor...
in the query.
"""
refcounts_before = self.query.alias_refcount.copy()
try:
extra_select, order_by, group_by = self.pre_s...
# This must come after 'select', 'ordering', ...
# docstring of get_from_clause() for details.
from_, f_params = self.get_from_clause()
where, w_params = self.compile(self.where) if...
having, h_params = self.compile(self.having) ...
params = []
result = ['SELECT']
out_cols = []
for _, (s_sql, s_params), alias in self.selec...
params.extend(s_params)
out_cols.append(s_sql)
result.append(', '.join(out_cols))
result.append('FROM')
result.extend(from_)
params.extend(f_params)
if where:
result.append('WHERE %s' % where)
params.extend(w_params)
# GROUP BY, HAVING, ORDER BYの処理
if with_limits:
if self.query.high_mark is not None:
result.append('LIMIT %d' % (self.quer...
if self.query.low_mark:
result.append('OFFSET %d' % self.quer...
return ' '.join(result), tuple(params)
finally:
# Finally do cleanup - get rid of the joins w...
self.query.reset_refcounts(refcounts_before)
}}
省略してもそこそこ長いですが、基本的には淡々とSELECT文を...
なお、今回については条件は設定してないのでwhereは空です。...
***pre_sql_setup [#wd56a184]
as_sqlメソッドではself.select等の設定が行われていません。...
#code(Python){{
def pre_sql_setup(self):
"""
Does any necessary class setup immediately prior ...
is for things that can't necessarily be done in _...
might not have all the pieces in place at that ti...
"""
self.setup_query()
order_by = self.get_order_by()
self.where, self.having = self.query.where.split_...
extra_select = self.get_extra_select(order_by, se...
group_by = self.get_group_by(self.select + extra_...
return extra_select, order_by, group_by
}}
さらにsetup_queryに進む。
#code(Python){{
def setup_query(self):
if all(self.query.alias_refcount[a] == 0 for a in...
self.query.get_initial_alias()
self.select, self.klass_info, self.annotation_col...
self.col_count = len(self.select)
}}
tablesは今まで出てきてないので空なはず。とはいうものの、a...
#code(Python){{
def get_initial_alias(self):
"""
Returns the first alias for this query, after inc...
count.
"""
if self.tables:
alias = self.tables[0]
self.ref_alias(alias)
else:
alias = self.join(BaseTable(self.get_meta().d...
return alias
}}
BaseTableはdjango.db.models.sql.datastructuresモジュール...
joinメソッド。といっても結合はしないのでそこら辺はばっさ...
#code(Python){{
def join(self, join, reuse=None):
alias, _ = self.table_alias(join.table_name, crea...
join.table_alias = alias
self.alias_map[alias] = join
return alias
}}
うーん、深い。もうそろそろ以下省略でもいいような気はしま...
#code(Python){{
def table_alias(self, table_name, create=False):
"""
Returns a table alias for the given table_name an...
new alias or not.
If 'create' is true, a new alias is always create...
most recently created alias for the table (if one...
"""
alias_list = self.table_map.get(table_name)
# Create a new alias for this table.
if alias_list:
# 省略
else:
# The first occurrence of a table uses the ta...
alias = table_name
self.table_map[alias] = [alias]
self.alias_refcount[alias] = 1
self.tables.append(alias)
return alias, True
}}
というわけで、aliasといいつつ今回はテーブル名そのものが返...
ここまでをまとめると、クエリーで使っているテーブル名をオ...
***get_select [#nef5a2aa]
さて、テーブル名設定するのに随分手間がかかりましたが（※実...
#code(Python){{
def get_select(self):
"""
Returns three values:
- a list of 3-tuples of (expression, (sql, params...
- a klass_info structure,
- a dictionary of annotations
The (sql, params) is what the expression will pro...
"AS alias" for the column (possibly None).
The klass_info structure contains the following i...
- Which model to instantiate
- Which columns for that model are present in the...
position of the select clause).
- related_klass_infos: [f, klass_info] to descent...
The annotations is a dictionary of {'attname': co...
"""
select = []
klass_info = None
annotations = {}
select_idx = 0
if self.query.default_cols:
select_list = []
for c in self.get_default_columns():
select_list.append(select_idx)
select.append((c, None))
select_idx += 1
klass_info = {
'model': self.query.model,
'select_fields': select_list,
}
ret = []
for col, alias in select:
ret.append((col, self.compile(col, select_for...
return ret, klass_info, annotations
}}
get_default_columnsメソッド。いつも通りに必要ないところは...
#code(Python){{
def get_default_columns(self, start_alias=None, opts=...
"""
Computes the default columns for selecting every ...
model. Will sometimes be called to pull in relate...
select_related), in which case "opts" and "start_...
to provide a starting point for the traversal.
"""
result = []
if opts is None:
opts = self.query.get_meta()
if not start_alias:
start_alias = self.query.get_initial_alias()
# The 'seen_models' is used to optimize checking ...
# alias for a given field. This also includes Non...
# be used by local fields.
seen_models = {None: start_alias}
for field in opts.concrete_fields:
model = field.model._meta.concrete_model
# A proxy model will have a different model a...
# will assign None if the field belongs to th...
if model == opts.model:
model = None
alias = self.query.join_parent_model(opts, mo...
seen_mod...
column = field.get_col(alias)
result.append(column)
return result
}}
queryのjoin_parent_modelはJOIN周りの処理を行っていますが...
fieldのget_colメソッド。場所はdjango/db/models/field/__in...
#code(Python){{
def get_col(self, alias, output_field=None):
if output_field is None:
output_field = self
if alias != self.model._meta.db_table or output_f...
from django.db.models.expressions import Col
return Col(alias, self, output_field)
else:
return self.cached_col
@cached_property
def cached_col(self):
from django.db.models.expressions import Col
return Col(self.model._meta.db_table, self)
}}
Col、また新しい概念が出てきました。メソッド内importにある...
#code(Python){{
class Col(Expression):
class Expression(BaseExpression, Combinable):
"""
An expression that can be combined with other express...
"""
class BaseExpression(object):
"""
Base class for all query expressions.
"""
class Combinable(object):
"""
Provides the ability to combine one or two objects with
some connector. For example F('foo') + F('bar').
"""
}}
うーん、どうやら式を表現するクラスでColとはColumnのことの...
再度SQLCompilerに戻って、compileメソッド
#code(Python){{
def compile(self, node, select_format=False):
vendor_impl = getattr(node, 'as_' + self.connecti...
if vendor_impl:
sql, params = vendor_impl(self, self.connecti...
else:
sql, params = node.as_sql(self, self.connecti...
if select_format and not self.subquery:
return node.output_field.select_format(self, ...
return sql, params
}}
nodeとは今回の場合Colオブジェクトです。compileメソッドはa...
***get_from_clause [#c423985a]
ようやくSELECTされる列が処理できたので次にFROMです。例に...
#code(Python){{
def get_from_clause(self):
"""
Returns a list of strings that are joined togethe...
"FROM" part of the query, as well as a list any e...
need to be included. Sub-classes, can override th...
from-clause via a "select".
This should only be called after any SQL construc...
might change the tables we need. This means the s...
ordering and distinct must be done first.
"""
result = []
params = []
for alias in self.query.tables:
try:
from_clause = self.query.alias_map[alias]
except KeyError:
# Extra tables can end up in self.tables,...
# alias_map if they aren't in a join. Tha...
continue
clause_sql, clause_params = self.compile(from...
result.append(clause_sql)
params.extend(clause_params)
return result, params
}}
alias_mapに入っているのは、大分前に見たからもう忘れ気味で...
BaseTableがcompileに渡されていますが（つまり、as_sqlが呼...
**cursor_iter [#s638ee22]
こんな感じにSQLが作られて、実行されて、実行後の処理です。...
#code(Python){{
def execute_sql(self, result_type=MULTI):
"""
Run the query against the database and returns th...
return value is a single data item if result_type...
iterator over the results if the result_type is M...
"""
try:
sql, params = self.as_sql()
except EmptyResultSet:
# 省略
cursor = self.connection.cursor()
try:
cursor.execute(sql, params)
except Exception:
# 省略
# result_typeがMULTI以外の処理
result = cursor_iter(
cursor, self.connection.features.empty_fetchm...
self.col_count
)
if not self.connection.features.can_use_chunked_r...
try:
# If we are using non-chunked reads, we r...
# structure as normally, but ensure it is...
# before going any further.
return list(result)
finally:
# done with the cursor
cursor.close()
return result
}}
cursor_iter。こいつは関数（正確にはジェネレータ）です。
#code(Python){{
def cursor_iter(cursor, sentinel, col_count):
"""
Yields blocks of rows from a cursor and ensures the c...
done.
"""
try:
for rows in iter((lambda: cursor.fetchmany(GET_IT...
sentinel):
yield [r[0:col_count] for r in rows]
finally:
cursor.close()
}}
えーっと、fetchmanyにより返されるのはリストのリスト（1レ...
**results_iter [#tb5e7404]
長かったexecute_sqlが終わり、どこまで戻ってくるかというと...
#code(Python){{
def __iter__(self):
queryset = self.queryset
db = queryset.db
compiler = queryset.query.get_compiler(using=db)
# Execute the query. This will also fill compiler...
# and annotations.
results = compiler.execute_sql()
select, klass_info, annotation_col_map = (compile...
compile...
model_cls = klass_info['model']
select_fields = klass_info['select_fields']
model_fields_start, model_fields_end = select_fie...
init_list = [f[0].target.attname
for f in select[model_fields_start:m...
related_populators = get_related_populators(klass...
for row in compiler.results_iter(results):
obj = model_cls.from_db(db, init_list, row[mo...
if related_populators:
for rel_populator in related_populators:
rel_populator.populate(row, obj)
if annotation_col_map:
for attr_name, col_pos in annotation_col_...
setattr(obj, attr_name, row[col_pos])
# Add the known related objects to the model,...
# 省略
yield obj
}}
results_iter。
#code(Python){{
def results_iter(self, results=None):
"""
Returns an iterator over the results from executi...
"""
converters = None
if results is None:
results = self.execute_sql(MULTI)
fields = [s[0] for s in self.select[0:self.col_co...
converters = self.get_converters(fields)
for rows in results:
for row in rows:
if converters:
row = self.apply_converters(row, conv...
yield row
}}
converterは実際動いているようですが、もうここまででいいで...
*おわりに [#rc7803c9]
今回はデータベースの華（？）、検索処理について見てきまし...
allの場合でもあくまで条件設定がされないだけで実行されるフ...
終了行:
[[Djangoを読む]]
#contents
*はじめに [#w3526026]
「APIで遊んでみる」の冒頭、saveメソッドが最終的にINSERTに...
その後、値を変更してもう一度saveメソッドを呼び出していま...
とすると次の読解対象は、
# objects.all() displays all the questions in the databa...
>>> Question.objects.all()
<QuerySet [<Question: Question object>]>
です。
*django/db/models/query.py [#dcb7ac74]
**QuerySet [#xef445b6]
objectsは前回も少し触れましたがManagerクラスのインスタン...
#code(Python){{
def all(self):
"""
Returns a new QuerySet that is a copy of the curr...
QuerySet to proxy for a model manager in some cas...
"""
return self._clone()
}}
シンプル。allメソッドを呼んだ時点ではQuerySetのコピーが作...
***__repr__ [#r5e3fa62]
シェルを使う場合、repr関数を使ってオブジェクトの表示が行...
#code(Python){{
def __repr__(self):
data = list(self[:REPR_OUTPUT_SIZE + 1])
if len(data) > REPR_OUTPUT_SIZE:
data[-1] = "...(remaining elements truncated)...
return '<QuerySet %r>' % data
}}
***__getitem__ [#i05b761a]
__getitem__に進む。初めに書いてある不正値チェックは省略。
#code(Python){{
def __getitem__(self, k):
"""
Retrieves an item or slice from the set of results.
"""
if self._result_cache is not None:
return self._result_cache[k]
if isinstance(k, slice):
qs = self._clone()
if k.start is not None:
start = int(k.start)
else:
start = None
if k.stop is not None:
stop = int(k.stop)
else:
stop = None
qs.query.set_limits(start, stop)
return list(qs)[::k.step] if k.step else qs
qs = self._clone()
qs.query.set_limits(k, k + 1)
return list(qs)[0]
}}
kはsliceです。で、
:start|None
:stop|REPR_OUTPUT_SIZE + 1 = 21
:step|None
です。
一応、set_limitsを見ておきましょう。queryはsqlモジュール...
#code(Python){{
def set_limits(self, low=None, high=None):
"""
Adjusts the limits on the rows retrieved. We use ...
as it makes it more Pythonic to read and write. W...
created, they are converted to the appropriate of...
Any limits passed in here are applied relative to...
constraints. So low is added to the current low v...
clamped to any existing high value.
"""
if high is not None:
if self.high_mark is not None:
self.high_mark = min(self.high_mark, self...
else:
self.high_mark = self.low_mark + high
if low is not None:
if self.high_mark is not None:
self.low_mark = min(self.high_mark, self....
else:
self.low_mark = self.low_mark + low
if self.low_mark == self.high_mark:
self.set_empty()
}}
無駄にややこしい(笑)。high_markの初期値はNone、low_markの...
:high_mark|21
:low_mark|0
という値が設定されます。
***__iter__ [#fbf8c40b]
さて、処理はQuerySetに返ってきます。stepは設定されてない...
#code(Python){{
def __repr__(self):
data = list(self[:REPR_OUTPUT_SIZE + 1])
if len(data) > REPR_OUTPUT_SIZE:
data[-1] = "...(remaining elements truncated)...
return '<QuerySet %r>' % data
}}
listはiterableを受け付けるので、__iter__が呼ばれるはずで...
#code(Python){{
def __iter__(self):
"""
The queryset iterator protocol uses three nested ...
default case:
1. sql.compiler:execute_sql()
- Returns 100 rows at time (constants.GET_...
using cursor.fetchmany(). This part is r...
doing some column masking, and returning...
2. sql/compiler.results_iter()
- Returns one row at time. At this point t...
tuples. In some cases the return values ...
Python values at this location.
3. self.iterator()
- Responsible for turning the rows into mo...
"""
self._fetch_all()
return iter(self._result_cache)
}}
コメントを見るとややこしそうですが、コードを見る限り、レ...
***_fetch_all [#o43dd085]
_fetch_allに進みます。
#code(Python){{
def _fetch_all(self):
if self._result_cache is None:
self._result_cache = list(self.iterator())
if self._prefetch_related_lookups and not self._p...
self._prefetch_related_objects()
}}
prefetchは無視。iteratorメソッドに進みます。
#code(Python){{
def iterator(self):
"""
An iterator over the results from applying this Q...
database.
"""
return iter(self._iterable_class(self))
}}
尻尾がつかめたと思ったらまた離れた感があります。_iterable...
**ModelIterable [#sb8a9d38]
ModelIterableクラスはdjango/db/models/query.pyの上の方に...
#code(Python){{
def __iter__(self):
queryset = self.queryset
db = queryset.db
compiler = queryset.query.get_compiler(using=db)
# Execute the query. This will also fill compiler...
# and annotations.
results = compiler.execute_sql()
select, klass_info, annotation_col_map = (compile...
compile...
model_cls = klass_info['model']
select_fields = klass_info['select_fields']
model_fields_start, model_fields_end = select_fie...
init_list = [f[0].target.attname
for f in select[model_fields_start:m...
related_populators = get_related_populators(klass...
for row in compiler.results_iter(results):
obj = model_cls.from_db(db, init_list, row[mo...
if related_populators:
for rel_populator in related_populators:
rel_populator.populate(row, obj)
if annotation_col_map:
for attr_name, col_pos in annotation_col_...
setattr(obj, attr_name, row[col_pos])
# Add the known related objects to the model,...
# 省略
yield obj
}}
+まずcompiler取得してSQL実行
+結果それぞれに対してモデルを取得、yieldで情報を渡す
ということが行われています。ぱっと見な感じだとrelated_pop...
*django.db.models.sql.compiler.SQLCompiler [#h81a1c13]
今回はqueryはQueryオブジェクトなので生成されるコンパイラ...
execute_sqlメソッドを見てみましょう。長いのでエラー処理と...
#code(Python){{
def execute_sql(self, result_type=MULTI):
"""
Run the query against the database and returns th...
return value is a single data item if result_type...
iterator over the results if the result_type is M...
"""
try:
sql, params = self.as_sql()
except EmptyResultSet:
# 省略
cursor = self.connection.cursor()
try:
cursor.execute(sql, params)
except Exception:
# 省略
# result_typeがMULTI以外の処理
result = cursor_iter(
cursor, self.connection.features.empty_fetchm...
self.col_count
)
if not self.connection.features.can_use_chunked_r...
try:
# If we are using non-chunked reads, we r...
# structure as normally, but ensure it is...
# before going any further.
return list(result)
finally:
# done with the cursor
cursor.close()
return result
}}
やっていることは、
+SQLを構築して
+実行して
+結果をイテレータとして返す
ということになります。ちなみに、sqlite3の場合、can_use_ch...
**as_sql [#ia7abe09]
as_sqlも長いので必要ないところは省略すると、
#code(Python){{
def as_sql(self, with_limits=True, with_col_aliases=F...
"""
Creates the SQL for this query. Returns the SQL s...
parameters.
If 'with_limits' is False, any limit/offset infor...
in the query.
"""
refcounts_before = self.query.alias_refcount.copy()
try:
extra_select, order_by, group_by = self.pre_s...
# This must come after 'select', 'ordering', ...
# docstring of get_from_clause() for details.
from_, f_params = self.get_from_clause()
where, w_params = self.compile(self.where) if...
having, h_params = self.compile(self.having) ...
params = []
result = ['SELECT']
out_cols = []
for _, (s_sql, s_params), alias in self.selec...
params.extend(s_params)
out_cols.append(s_sql)
result.append(', '.join(out_cols))
result.append('FROM')
result.extend(from_)
params.extend(f_params)
if where:
result.append('WHERE %s' % where)
params.extend(w_params)
# GROUP BY, HAVING, ORDER BYの処理
if with_limits:
if self.query.high_mark is not None:
result.append('LIMIT %d' % (self.quer...
if self.query.low_mark:
result.append('OFFSET %d' % self.quer...
return ' '.join(result), tuple(params)
finally:
# Finally do cleanup - get rid of the joins w...
self.query.reset_refcounts(refcounts_before)
}}
省略してもそこそこ長いですが、基本的には淡々とSELECT文を...
なお、今回については条件は設定してないのでwhereは空です。...
***pre_sql_setup [#wd56a184]
as_sqlメソッドではself.select等の設定が行われていません。...
#code(Python){{
def pre_sql_setup(self):
"""
Does any necessary class setup immediately prior ...
is for things that can't necessarily be done in _...
might not have all the pieces in place at that ti...
"""
self.setup_query()
order_by = self.get_order_by()
self.where, self.having = self.query.where.split_...
extra_select = self.get_extra_select(order_by, se...
group_by = self.get_group_by(self.select + extra_...
return extra_select, order_by, group_by
}}
さらにsetup_queryに進む。
#code(Python){{
def setup_query(self):
if all(self.query.alias_refcount[a] == 0 for a in...
self.query.get_initial_alias()
self.select, self.klass_info, self.annotation_col...
self.col_count = len(self.select)
}}
tablesは今まで出てきてないので空なはず。とはいうものの、a...
#code(Python){{
def get_initial_alias(self):
"""
Returns the first alias for this query, after inc...
count.
"""
if self.tables:
alias = self.tables[0]
self.ref_alias(alias)
else:
alias = self.join(BaseTable(self.get_meta().d...
return alias
}}
BaseTableはdjango.db.models.sql.datastructuresモジュール...
joinメソッド。といっても結合はしないのでそこら辺はばっさ...
#code(Python){{
def join(self, join, reuse=None):
alias, _ = self.table_alias(join.table_name, crea...
join.table_alias = alias
self.alias_map[alias] = join
return alias
}}
うーん、深い。もうそろそろ以下省略でもいいような気はしま...
#code(Python){{
def table_alias(self, table_name, create=False):
"""
Returns a table alias for the given table_name an...
new alias or not.
If 'create' is true, a new alias is always create...
most recently created alias for the table (if one...
"""
alias_list = self.table_map.get(table_name)
# Create a new alias for this table.
if alias_list:
# 省略
else:
# The first occurrence of a table uses the ta...
alias = table_name
self.table_map[alias] = [alias]
self.alias_refcount[alias] = 1
self.tables.append(alias)
return alias, True
}}
というわけで、aliasといいつつ今回はテーブル名そのものが返...
ここまでをまとめると、クエリーで使っているテーブル名をオ...
***get_select [#nef5a2aa]
さて、テーブル名設定するのに随分手間がかかりましたが（※実...
#code(Python){{
def get_select(self):
"""
Returns three values:
- a list of 3-tuples of (expression, (sql, params...
- a klass_info structure,
- a dictionary of annotations
The (sql, params) is what the expression will pro...
"AS alias" for the column (possibly None).
The klass_info structure contains the following i...
- Which model to instantiate
- Which columns for that model are present in the...
position of the select clause).
- related_klass_infos: [f, klass_info] to descent...
The annotations is a dictionary of {'attname': co...
"""
select = []
klass_info = None
annotations = {}
select_idx = 0
if self.query.default_cols:
select_list = []
for c in self.get_default_columns():
select_list.append(select_idx)
select.append((c, None))
select_idx += 1
klass_info = {
'model': self.query.model,
'select_fields': select_list,
}
ret = []
for col, alias in select:
ret.append((col, self.compile(col, select_for...
return ret, klass_info, annotations
}}
get_default_columnsメソッド。いつも通りに必要ないところは...
#code(Python){{
def get_default_columns(self, start_alias=None, opts=...
"""
Computes the default columns for selecting every ...
model. Will sometimes be called to pull in relate...
select_related), in which case "opts" and "start_...
to provide a starting point for the traversal.
"""
result = []
if opts is None:
opts = self.query.get_meta()
if not start_alias:
start_alias = self.query.get_initial_alias()
# The 'seen_models' is used to optimize checking ...
# alias for a given field. This also includes Non...
# be used by local fields.
seen_models = {None: start_alias}
for field in opts.concrete_fields:
model = field.model._meta.concrete_model
# A proxy model will have a different model a...
# will assign None if the field belongs to th...
if model == opts.model:
model = None
alias = self.query.join_parent_model(opts, mo...
seen_mod...
column = field.get_col(alias)
result.append(column)
return result
}}
queryのjoin_parent_modelはJOIN周りの処理を行っていますが...
fieldのget_colメソッド。場所はdjango/db/models/field/__in...
#code(Python){{
def get_col(self, alias, output_field=None):
if output_field is None:
output_field = self
if alias != self.model._meta.db_table or output_f...
from django.db.models.expressions import Col
return Col(alias, self, output_field)
else:
return self.cached_col
@cached_property
def cached_col(self):
from django.db.models.expressions import Col
return Col(self.model._meta.db_table, self)
}}
Col、また新しい概念が出てきました。メソッド内importにある...
#code(Python){{
class Col(Expression):
class Expression(BaseExpression, Combinable):
"""
An expression that can be combined with other express...
"""
class BaseExpression(object):
"""
Base class for all query expressions.
"""
class Combinable(object):
"""
Provides the ability to combine one or two objects with
some connector. For example F('foo') + F('bar').
"""
}}
うーん、どうやら式を表現するクラスでColとはColumnのことの...
再度SQLCompilerに戻って、compileメソッド
#code(Python){{
def compile(self, node, select_format=False):
vendor_impl = getattr(node, 'as_' + self.connecti...
if vendor_impl:
sql, params = vendor_impl(self, self.connecti...
else:
sql, params = node.as_sql(self, self.connecti...
if select_format and not self.subquery:
return node.output_field.select_format(self, ...
return sql, params
}}
nodeとは今回の場合Colオブジェクトです。compileメソッドはa...
***get_from_clause [#c423985a]
ようやくSELECTされる列が処理できたので次にFROMです。例に...
#code(Python){{
def get_from_clause(self):
"""
Returns a list of strings that are joined togethe...
"FROM" part of the query, as well as a list any e...
need to be included. Sub-classes, can override th...
from-clause via a "select".
This should only be called after any SQL construc...
might change the tables we need. This means the s...
ordering and distinct must be done first.
"""
result = []
params = []
for alias in self.query.tables:
try:
from_clause = self.query.alias_map[alias]
except KeyError:
# Extra tables can end up in self.tables,...
# alias_map if they aren't in a join. Tha...
continue
clause_sql, clause_params = self.compile(from...
result.append(clause_sql)
params.extend(clause_params)
return result, params
}}
alias_mapに入っているのは、大分前に見たからもう忘れ気味で...
BaseTableがcompileに渡されていますが（つまり、as_sqlが呼...
**cursor_iter [#s638ee22]
こんな感じにSQLが作られて、実行されて、実行後の処理です。...
#code(Python){{
def execute_sql(self, result_type=MULTI):
"""
Run the query against the database and returns th...
return value is a single data item if result_type...
iterator over the results if the result_type is M...
"""
try:
sql, params = self.as_sql()
except EmptyResultSet:
# 省略
cursor = self.connection.cursor()
try:
cursor.execute(sql, params)
except Exception:
# 省略
# result_typeがMULTI以外の処理
result = cursor_iter(
cursor, self.connection.features.empty_fetchm...
self.col_count
)
if not self.connection.features.can_use_chunked_r...
try:
# If we are using non-chunked reads, we r...
# structure as normally, but ensure it is...
# before going any further.
return list(result)
finally:
# done with the cursor
cursor.close()
return result
}}
cursor_iter。こいつは関数（正確にはジェネレータ）です。
#code(Python){{
def cursor_iter(cursor, sentinel, col_count):
"""
Yields blocks of rows from a cursor and ensures the c...
done.
"""
try:
for rows in iter((lambda: cursor.fetchmany(GET_IT...
sentinel):
yield [r[0:col_count] for r in rows]
finally:
cursor.close()
}}
えーっと、fetchmanyにより返されるのはリストのリスト（1レ...
**results_iter [#tb5e7404]
長かったexecute_sqlが終わり、どこまで戻ってくるかというと...
#code(Python){{
def __iter__(self):
queryset = self.queryset
db = queryset.db
compiler = queryset.query.get_compiler(using=db)
# Execute the query. This will also fill compiler...
# and annotations.
results = compiler.execute_sql()
select, klass_info, annotation_col_map = (compile...
compile...
model_cls = klass_info['model']
select_fields = klass_info['select_fields']
model_fields_start, model_fields_end = select_fie...
init_list = [f[0].target.attname
for f in select[model_fields_start:m...
related_populators = get_related_populators(klass...
for row in compiler.results_iter(results):
obj = model_cls.from_db(db, init_list, row[mo...
if related_populators:
for rel_populator in related_populators:
rel_populator.populate(row, obj)
if annotation_col_map:
for attr_name, col_pos in annotation_col_...
setattr(obj, attr_name, row[col_pos])
# Add the known related objects to the model,...
# 省略
yield obj
}}
results_iter。
#code(Python){{
def results_iter(self, results=None):
"""
Returns an iterator over the results from executi...
"""
converters = None
if results is None:
results = self.execute_sql(MULTI)
fields = [s[0] for s in self.select[0:self.col_co...
converters = self.get_converters(fields)
for rows in results:
for row in rows:
if converters:
row = self.apply_converters(row, conv...
yield row
}}
converterは実際動いているようですが、もうここまででいいで...
*おわりに [#rc7803c9]
今回はデータベースの華（？）、検索処理について見てきまし...
allの場合でもあくまで条件設定がされないだけで実行されるフ...
ページ名:
![[PukiWiki]](image/pukiwiki.png "[PukiWiki]")
