web-dev-qa-db-ja.com

Sklearn前処理-PolynomialFeatures-出力配列/データフレームの列名/ヘッダーを保持する方法

TLDR：sklearn.preprocessing.PolynomialFeatures（）関数から出力numpy配列のヘッダーを取得する方法

次のコードがあるとしましょう...

import pandas as pd
import numpy as np
from sklearn import preprocessing as pp

a = np.ones(3)
b = np.ones(3) * 2
c = np.ones(3) * 3

input_df = pd.DataFrame([a,b,c])
input_df = input_df.T
input_df.columns=['a', 'b', 'c']

input_df

    a   b   c
0   1   2   3
1   1   2   3
2   1   2   3

poly = pp.PolynomialFeatures(2)
output_nparray = poly.fit_transform(input_df)
print output_nparray

[[ 1.  1.  2.  3.  1.  2.  3.  4.  6.  9.]
 [ 1.  1.  2.  3.  1.  2.  3.  4.  6.  9.]
 [ 1.  1.  2.  3.  1.  2.  3.  4.  6.  9.]]

3x10マトリックス/ output_nparrayを取得して、a、b、cラベルを上記のデータにどのように関連付けるかをどのように引き継ぐことができますか？

pythonpython-2.7validationscikit-learncross-validation
15
Afflatus

実例、すべて1行で（ここでは、「読みやすさ」は目標ではないと想定しています）：

target_feature_names = ['x'.join(['{}^{}'.format(pair[0],pair[1]) for pair in Tuple if pair[1]!=0]) for Tuple in [Zip(input_df.columns,p) for p in poly.powers_]]
output_df = pd.DataFrame(output_nparray, columns = target_feature_names)

Update：@OmerBが指摘したように、今ではget_feature_namesメソッド ：

>> poly.get_feature_names(input_df.columns)
['1', 'a', 'b', 'c', 'a^2', 'a b', 'a c', 'b^2', 'b c', 'c^2']
13
Guiem Bosch

scikit-learn 0.18に気の利いた get_feature_names() メソッドが追加されました！

>> input_df.columns
Index(['a', 'b', 'c'], dtype='object')

>> poly.fit_transform(input_df)
array([[ 1.,  1.,  2.,  3.,  1.,  2.,  3.,  4.,  6.,  9.],
       [ 1.,  1.,  2.,  3.,  1.,  2.,  3.,  4.,  6.,  9.],
       [ 1.,  1.,  2.,  3.,  1.,  2.,  3.,  4.,  6.,  9.]])

>> poly.get_feature_names(input_df.columns)
['1', 'a', 'b', 'c', 'a^2', 'a b', 'a c', 'b^2', 'b c', 'c^2']

Sklearnはそれ自体でDataFrameから読み取らないため、列名を指定する必要があることに注意してください。

8
OmerB

これは機能します：

def PolynomialFeatures_labeled(input_df,power):
    '''Basically this is a cover for the sklearn preprocessing function. 
    The problem with that function is if you give it a labeled dataframe, it ouputs an unlabeled dataframe with potentially
    a whole bunch of unlabeled columns. 

    Inputs:
    input_df = Your labeled pandas dataframe (list of x's not raised to any power) 
    power = what order polynomial you want variables up to. (use the same power as you want entered into pp.PolynomialFeatures(power) directly)

    Ouput:
    Output: This function relies on the powers_ matrix which is one of the preprocessing function's outputs to create logical labels and 
    outputs a labeled pandas dataframe   
    '''
    poly = pp.PolynomialFeatures(power)
    output_nparray = poly.fit_transform(input_df)
    powers_nparray = poly.powers_

    input_feature_names = list(input_df.columns)
    target_feature_names = ["Constant Term"]
    for feature_distillation in powers_nparray[1:]:
        intermediary_label = ""
        final_label = ""
        for i in range(len(input_feature_names)):
            if feature_distillation[i] == 0:
                continue
            else:
                variable = input_feature_names[i]
                power = feature_distillation[i]
                intermediary_label = "%s^%d" % (variable,power)
                if final_label == "":         #If the final label isn't yet specified
                    final_label = intermediary_label
                else:
                    final_label = final_label + " x " + intermediary_label
        target_feature_names.append(final_label)
    output_df = pd.DataFrame(output_nparray, columns = target_feature_names)
    return output_df

output_df = PolynomialFeatures_labeled(input_df,2)
output_df

    Constant Term   a^1 b^1 c^1 a^2 a^1 x b^1   a^1 x c^1   b^2 b^1 x c^1   c^2
0               1   1   2   3   1           2           3   4           6   9
1               1   1   2   3   1           2           3   4           6   9
2               1   1   2   3   1           2           3   4           6   9
2
Afflatus

get_feature_names()メソッドは適切ですが、すべての変数を_'x1'_、_'x2'_、_'x1 x2'_などとして返します。以下は、get_feature_names()出力を_'Col_1'_、_'Col_2'_、_'Col_1 x Col_2'_としてフォーマットされた列名のリストにすばやく変換する関数です。

に：

_def PolynomialFeatureNames(sklearn_feature_name_output, df):
"""
This function takes the output from the .get_feature_names() method on the PolynomialFeatures 
instance and replaces values with df column names to return output such as 'Col_1 x Col_2'

sklearn_feature_name_output: The list object returned when calling .get_feature_names() on the PolynomialFeatures object
df: Pandas dataframe with correct column names
"""
import re
cols = df.columns.tolist()
feat_map = {'x'+str(num):cat for num, cat in enumerate(cols)}
feat_string = ','.join(sklearn_feature_name_output)
for k,v in feat_map.items():
    feat_string = re.sub(fr"\b{k}\b",v,feat_string)
return feat_string.replace(" "," x ").split(',')  

interaction = PolynomialFeatures(degree=2)
X_inter = interaction.fit_transform(input_df)

names = PolynomialFeatureNames(interaction.get_feature_names(),input_df)

print(pd.DataFrame(X_inter, columns= names))
_

でる：

_            1       a       b       c     a^2   a x b   a x c     b^2   b x c  \
0 1.00000 1.00000 2.00000 3.00000 1.00000 2.00000 3.00000 4.00000 6.00000   
1 1.00000 1.00000 2.00000 3.00000 1.00000 2.00000 3.00000 4.00000 6.00000   
2 1.00000 1.00000 2.00000 3.00000 1.00000 2.00000 3.00000 4.00000 6.00000   

      c^2  
0 9.00000  
1 9.00000  
2 9.00000
_
0
Negative Correlation