v0.8.3

- add DCN-M(DCN V2) and DCN-Mix model - add transform_fn for DenseFeat - simplify methods in `interaction.py` - change output shape of BilinearInteraction used in FiBiNET - update docs and test files
shenweichen · Jan 6, 2021 · 43b971c · 43b971c
2 parents e9c8f08 + 8360dc7
commit 43b971c
Show file tree

Hide file tree

Showing 57 changed files with 435 additions and 96 deletions.
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -19,8 +19,8 @@ Steps to reproduce the behavior:
 
 **Operating environment(运行环境):**
  - python version [e.g. 3.5, 3.7]
- - tensorflow version [e.g. 1.4.0, 1.15.0, 2.3.0]
- - deepctr version [e.g. 0.8.2,]
+ - tensorflow version [e.g. 1.4.0, 1.15.0, 2.4.0]
+ - deepctr version [e.g. 0.8.3,]
 
 **Additional context**
 Add any other context about the problem here.
diff --git a/.github/ISSUE_TEMPLATE/question.md b/.github/ISSUE_TEMPLATE/question.md
@@ -16,5 +16,5 @@ Add any other context about the problem here.
 
 **Operating environment(运行环境):**
  - python version [e.g. 3.6]
- - tensorflow version [e.g. 1.4.0, 1.5.0, 2.3.0]
- - deepctr version [e.g. 0.8.2,]
+ - tensorflow version [e.g. 1.4.0, 1.5.0, 2.4.0]
+ - deepctr version [e.g. 0.8.3,]
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -18,7 +18,7 @@ jobs:
     strategy:
       matrix:
         python-version: [3.6,3.7]
-        tf-version: [1.4.0,1.15.0,2.1.0,2.3.0]
+        tf-version: [1.4.0,1.15.0,2.1.0,2.4.0]
 
         exclude:
           - python-version: 3.7

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,10 +1,21 @@
 This project is under development and we need developers to participate in.
-
+# Join us
 If you
 
 - familiar with and interested in ctr prediction algorithms
 - familiar with tensorflow 
 - have spare time to learn and develop
 - familiar with git
 
-please send a brief introduction of your background and experience to [email protected], welcome to join us!
+please send a brief introduction of your background and experience to [email protected], welcome to join us!
+
+# Creating a pull request
+1. **Become a collaborator**: Send an email with introduction and your github account name to [email protected] and waiting for invitation to become a collaborator.
+2. **Fork&Dev**: Fork your own branch(`dev_yourname`) in `DeepCTR` from the `master` branch for development.If the `master` is updated during the development process, remember to merge and update to `dev_yourname` regularly.
+3. **Testing**: Test  logical correctness and effect when finishing the code development of the `dev_yourname` branch. 
+4. **Pre-release** : After testing contact [email protected] for pre-release integration, usually your branch `dev_yourname` will be merged into `release` branch by squash merge. 
+5. **Release a new version**: After confirming that the change is no longer needed, `release` branch will be merged into `master` and a new python package will be released on pypi.
+
+# Discussions
+
+https://github.com/shenweichen/DeepCTR/discussions
diff --git a/README.md b/README.md
@@ -10,7 +10,7 @@
 
 
 [![Documentation Status](https://readthedocs.org/projects/deepctr-doc/badge/?version=latest)](https://deepctr-doc.readthedocs.io/)
-[![Build Status](https://travis-ci.org/shenweichen/DeepCTR.svg?branch=master)](https://travis-ci.org/shenweichen/DeepCTR)
+![CI status](https://github.com/shenweichen/deepctr/workflows/CI/badge.svg)
 [![Coverage Status](https://coveralls.io/repos/github/shenweichen/DeepCTR/badge.svg?branch=master)](https://coveralls.io/github/shenweichen/DeepCTR?branch=master)
 [![Codacy Badge](https://api.codacy.com/project/badge/Grade/d4099734dc0e4bab91d332ead8c0bdd0)](https://www.codacy.com/app/wcshen1994/DeepCTR?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=shenweichen/DeepCTR&amp;utm_campaign=Badge_Grade)
 [![Disscussion](https://img.shields.io/badge/chat-wechat-brightgreen?style=flat)](./README.md#disscussiongroup)
@@ -45,15 +45,16 @@ Let's [**Get Started!**](https://deepctr-doc.readthedocs.io/en/latest/Quick-Star
 |   Attentional Factorization Machine    | [IJCAI 2017][Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](http://www.ijcai.org/proceedings/2017/435) |
 |      Neural Factorization Machine      | [SIGIR 2017][Neural Factorization Machines for Sparse Predictive Analytics](https://arxiv.org/pdf/1708.05027.pdf)                                               |
 |                xDeepFM                 | [KDD 2018][xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://arxiv.org/pdf/1803.05170.pdf)                         |
-|                AutoInt                 | [arxiv 2018][AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://arxiv.org/abs/1810.11921)                              |
-|         Deep Interest Network          | [KDD 2018][Deep Interest Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1706.06978.pdf)                                                       |
+|         Deep Interest Network          | [KDD 2018][Deep Interest Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1706.06978.pdf)     
+|                AutoInt                 | [CIKM 2019][AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://arxiv.org/abs/1810.11921)                              ||
 |    Deep Interest Evolution Network     | [AAAI 2019][Deep Interest Evolution Network for Click-Through Rate Prediction](https://arxiv.org/pdf/1809.03672.pdf)                                            |
 |                FwFM                    | [WWW 2018][Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising](https://arxiv.org/pdf/1806.03514.pdf)                |
 |                  ONN                  | [arxiv 2019][Operation-aware Neural Networks for User Response Prediction](https://arxiv.org/pdf/1904.12579.pdf)                                                |
 |                 FGCNN                  | [WWW 2019][Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction ](https://arxiv.org/pdf/1904.04447)                             |
 |     Deep Session Interest Network      | [IJCAI 2019][Deep Session Interest Network for Click-Through Rate Prediction ](https://arxiv.org/abs/1905.06482)                                                |
 |                FiBiNET                 | [RecSys 2019][FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction](https://arxiv.org/pdf/1905.09433.pdf)   |
 |                FLEN                    | [arxiv 2019][FLEN: Leveraging Field for Scalable CTR Prediction](https://arxiv.org/pdf/1911.04690.pdf)   |
+|                DCN V2                    | [arxiv 2020][DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://arxiv.org/abs/2008.13535)   |
 
 ## Citation
 
@@ -75,7 +76,7 @@ If you find this code useful in your research, please cite it using the followin
 
 ## DisscussionGroup 交流群
 
-Please follow our wechat to join group:  
+- [Discussions](https://github.com/shenweichen/DeepCTR/discussions)
 - 公众号：**浅梦的学习笔记**  
 - wechat ID: **deepctrbot**
 

diff --git a/deepctr/__init__.py b/deepctr/__init__.py
@@ -1,4 +1,4 @@
 from .utils import check_version
 
-__version__ = '0.8.2'
+__version__ = '0.8.3'
 check_version(__version__)
diff --git a/deepctr/estimator/models/afm.py b/deepctr/estimator/models/afm.py
@@ -2,7 +2,7 @@
 """
 
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Xiao J, Ye H, He X, et al. Attentional factorization machines: Learning the weight of feature interactions via attention networks[J]. arXiv preprint arXiv:1708.04617, 2017.

diff --git a/deepctr/estimator/models/autoint.py b/deepctr/estimator/models/autoint.py
@@ -2,7 +2,7 @@
 """
 
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Song W, Shi C, Xiao Z, et al. AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks[J]. arXiv preprint arXiv:1810.11921, 2018.(https://arxiv.org/abs/1810.11921)

diff --git a/deepctr/estimator/models/ccpm.py b/deepctr/estimator/models/ccpm.py
@@ -2,7 +2,7 @@
 """
 
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Liu Q, Yu F, Wu S, et al. A convolutional click prediction model[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 2015: 1743-1746.

diff --git a/deepctr/estimator/models/dcn.py b/deepctr/estimator/models/dcn.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Wang R, Fu B, Fu G, et al. Deep & cross network for ad click predictions[C]//Proceedings of the ADKDD'17. ACM, 2017: 12. (https://arxiv.org/abs/1708.05123)

diff --git a/deepctr/estimator/models/deepfm.py b/deepctr/estimator/models/deepfm.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Guo H, Tang R, Ye Y, et al. Deepfm: a factorization-machine based neural network for ctr prediction[J]. arXiv preprint arXiv:1703.04247, 2017.(https://arxiv.org/abs/1703.04247)

diff --git a/deepctr/estimator/models/fibinet.py b/deepctr/estimator/models/fibinet.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Huang T, Zhang Z, Zhang J. FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction[J]. arXiv preprint arXiv:1905.09433, 2019.

diff --git a/deepctr/estimator/models/fnn.py b/deepctr/estimator/models/fnn.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Zhang W, Du T, Wang J. Deep learning over multi-field categorical data[C]//European conference on information retrieval. Springer, Cham, 2016: 45-57.(https://arxiv.org/pdf/1601.02376.pdf)

diff --git a/deepctr/estimator/models/fwfm.py b/deepctr/estimator/models/fwfm.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
     Harshit Pande
 
 Reference:

diff --git a/deepctr/estimator/models/nfm.py b/deepctr/estimator/models/nfm.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] He X, Chua T S. Neural factorization machines for sparse predictive analytics[C]//Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 2017: 355-364. (https://arxiv.org/abs/1708.05027)

diff --git a/deepctr/estimator/models/pnn.py b/deepctr/estimator/models/pnn.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Qu Y, Cai H, Ren K, et al. Product-based neural networks for user response prediction[C]//Data Mining (ICDM), 2016 IEEE 16th International Conference on. IEEE, 2016: 1149-1154.(https://arxiv.org/pdf/1611.00144.pdf)

diff --git a/deepctr/estimator/models/wdl.py b/deepctr/estimator/models/wdl.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Cheng H T, Koc L, Harmsen J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016: 7-10.(https://arxiv.org/pdf/1606.07792.pdf)

diff --git a/deepctr/estimator/models/xdeepfm.py b/deepctr/estimator/models/xdeepfm.py
@@ -1,7 +1,7 @@
 # -*- coding:utf-8 -*-
 """
 Author:
-    Weichen Shen,[email protected]
+    Weichen Shen, [email protected]
 
 Reference:
     [1] Lian J, Zhou X, Zhang F, et al. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems[J]. arXiv preprint arXiv:1803.05170, 2018.(https://arxiv.org/pdf/1803.05170.pdf)

diff --git a/deepctr/feature_column.py b/deepctr/feature_column.py
@@ -87,11 +87,21 @@ def __hash__(self):
         return self.name.__hash__()
 
 
-class DenseFeat(namedtuple('DenseFeat', ['name', 'dimension', 'dtype'])):
+class DenseFeat(namedtuple('DenseFeat', ['name', 'dimension', 'dtype', 'transform_fn'])):
+    """ Dense feature
+    Args:
+        name: feature name,
+        dimension: dimension of the feature, default = 1.
+        dtype: dtype of the feature, default="float32".
+        transform_fn: If not None, a function that can be used to transfrom
+        values of the feature.  the function takes the input Tensor as its
+        argument, and returns the output Tensor. 
+        (e.g. lambda x: (x - 3.0) / 4.2).
+    """
     __slots__ = ()
 
-    def __new__(cls, name, dimension=1, dtype="float32"):
-        return super(DenseFeat, cls).__new__(cls, name, dimension, dtype)
+    def __new__(cls, name, dimension=1, dtype="float32", transform_fn=None):
+        return super(DenseFeat, cls).__new__(cls, name, dimension, dtype, transform_fn)
 
     def __hash__(self):
         return self.name.__hash__()

diff --git a/deepctr/inputs.py b/deepctr/inputs.py
@@ -9,7 +9,7 @@
 from collections import defaultdict
 from itertools import chain
 
-from tensorflow.python.keras.layers import Embedding
+from tensorflow.python.keras.layers import Embedding, Lambda
 from tensorflow.python.keras.regularizers import l2
 
 from .layers.sequence import SequencePoolingLayer, WeightedSequenceLayer
@@ -138,7 +138,11 @@ def get_dense_input(features, feature_columns):
         filter(lambda x: isinstance(x, fc_lib.DenseFeat), feature_columns)) if feature_columns else []
     dense_input_list = []
     for fc in dense_feature_columns:
-        dense_input_list.append(features[fc.name])
+        if fc.transform_fn is None:
+            dense_input_list.append(features[fc.name])
+        else:
+            transform_result = Lambda(fc.transform_fn)(features[fc.name])
+            dense_input_list.append(transform_result)
     return dense_input_list
 
 

diff --git a/deepctr/layers/__init__.py b/deepctr/layers/__init__.py
@@ -2,7 +2,7 @@
 
 from .activation import Dice
 from .core import DNN, LocalActivationUnit, PredictionLayer
-from .interaction import (CIN, FM, AFMLayer, BiInteractionPooling, CrossNet,
+from .interaction import (CIN, FM, AFMLayer, BiInteractionPooling, CrossNet, CrossNetMix,
                           InnerProductLayer, InteractingLayer,
                           OutterProductLayer, FGCNNLayer, SENETLayer, BilinearInteraction,
                           FieldWiseBiInteraction, FwFMLayer)
@@ -20,6 +20,7 @@
                   'FM': FM,
                   'AFMLayer': AFMLayer,
                   'CrossNet': CrossNet,
+                  'CrossNetMix': CrossNetMix,
                   'BiInteractionPooling': BiInteractionPooling,
                   'LocalActivationUnit': LocalActivationUnit,
                   'Dice': Dice,