azure / counterfit Goto Github PK

Describe
Contact email [email protected] does not exist and emails sent to it are undeliverable. Sent from @tylermneher and CC’d on it we’re @westover and @thinkula - all three of us members of @neherdata

To Reproduce
Steps to reproduce the behavior:

1. Send an email to [email protected] as is listed on the bottom of README.md
2. See error upon email bounce back.

Expected behavior
Email to be delivered to listed address

Stack trace

<div class="x-apple-transform-wrapper" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-tap-highlight-color: rgba(26, 26, 26, 0.3); -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: UICTFontTextStyleBody; font-size: 15px; background-color: rgb(255, 255, 255); height: 1228.065693px;">

Original Message Details
--
Created Date:12/25/2022 12:14:06 PMSender Address:[email protected] Address:[email protected]:request for more information | Created Date: | 12/25/2022 12:14:06 PM | Sender Address: | [email protected] | Recipient Address: | [email protected] | Subject: | request for more information
Created Date: | 12/25/2022 12:14:06 PM
Sender Address: | [email protected]
Recipient Address: | [email protected]
Subject: | request for more information
Error Details
Reported error:550 5.4.1 Recipient address rejected: Access denied. AS(201806281) [BL2NAM06FT011.Eop-nam06.prod.protection.outlook.com]DSN generated by:DM4PR11MB8227.namprd11.prod.outlook.com | Reported error: | 550 5.4.1 Recipient address rejected: Access denied. AS(201806281) [BL2NAM06FT011.Eop-nam06.prod.protection.outlook.com] | DSN generated by: | DM4PR11MB8227.namprd11.prod.outlook.com
Reported error: | 550 5.4.1 Recipient address rejected: Access denied. AS(201806281) [BL2NAM06FT011.Eop-nam06.prod.protection.outlook.com]
DSN generated by: | DM4PR11MB8227.namprd11.prod.outlook.com

</div><p style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-tap-highlight-color: rgba(26, 26, 26, 0.3); -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: &quot;Segoe UI&quot;, Frutiger, Arial, sans-serif; font-size: 17px; font-weight: 500; padding-top: 4px; padding-bottom: 0px; margin-top: 19px; margin-bottom: 5px;">Original Message Headers</p><pre style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-tap-highlight-color: rgba(26, 26, 26, 0.3); -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-size: 15px; background-color: rgb(255, 255, 255); color: gray; white-space: pre; padding-top: 0px; margin-top: 5px;">ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=<span dir="ltr">microsoft.com</span>; cv=none;
 b=UaLcfCXSy8HHoaiEeD5WFRXs+fX36Rn6FVODAW0y4zf/aQpPZEEmshl0YKZEW3yGfozUTwM8PqLToBfxP7V3yYeCwMn/BceKmTZb4fJhiECwOQYvu5hMajmfqk5DIJS0dls1OLF5HN8hWrUuAvpMaspAt3XOysmKG/GQvizvV0ct5O7DfhwxUKaR95e30X43AKglExaW0+VvAH/LPpitQzjYIyWijjhdtP9dYFo45u8vvpBDbkWU6XFpRqqY0mxDqaBJpH7apugpetZvhgtKQ5zPfEfKyCwotLxOZiX5Omla6D2Ed4qxjzU4s+kimX7tOdi4Ttq+ruclAmcSmpj5qg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=<span dir="ltr">microsoft.com</span>;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OmMM8fx8TwkfRSb1jK2MEtTleiMqWi6afyXfaacIKSs=;
 b=cdcXhaAOHz6LOxPTiyDctZSs15e/1luclRnkXde04GAdeu9EIaDLzyj1bs7vDzxETZMRbnv0JW8gQZIvjG8n9aqwQGkld3CB8QtDSfTrBuezJGNxq0/Zgb2J7nW0rxr4wFfn89tkjeZxS0d/Tr+QdiDajzUbIz2G91lBvYW7kwlqLZjQAoZcqEC+g5K34i80vbicyGbcgqmj5n6mhBCJqKh1lrTRYPFI4q6mvJbt9QXUGyy9B2TheIAlnVFZYHw5dpl9ZDTjUPq4TzHTp/j1glruZfH0mu0gwCtxL8zxOGM5RJqwuBh+yD0PbXqOQYojwt6jIcrz1Sf8mIlDXjT3Jg==
ARC-Authentication-Results: i=1; <span dir="ltr">mx.microsoft.com</span> 1; spf=pass
 smtp.mailfrom=<span dir="ltr">neherdata.com</span>; dmarc=pass action=none
 header.from=<span dir="ltr">neherdata.com</span>; dkim=pass header.d=<span dir="ltr">neherdata.com</span>; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=<span dir="ltr">neherdata.com</span>;
 s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OmMM8fx8TwkfRSb1jK2MEtTleiMqWi6afyXfaacIKSs=;
 b=N9HOraER/+KYu3bJ0EQsAHcWdAu3Z800yqCcBJmvv1NKx20r/K2dB/thPjTPduy8KqahSReorhfSu9UaPhIKlrlugvCaCWfGW1nGvxl6NUaXly/7BbKZR4vCgMiHCawVdrPiTlOw+4E3U2hNNwIsnwVSpnFFTYTcPlJH8kQ1jOkX2zeGk7jMRG586k8QLzQFNY6IVEWf9V6Z6SaTzq+1Z7HB5JHnl+Y1VwjphwMQ4K+spn0A1WF6g+ooXic5atT+5hUd8xwwfcFk4UfirLgxL4A0E9DsKaNXwXfJdJDdybsvexlUNOxReKHah1tfEHv8TGOl7lpJYfZ7hNz7+19cGg==
Received: from <span dir="ltr">DM8PR11MB5653.namprd11.prod.outlook.com</span> (2603:10b6:8:25::8) by
 <span dir="ltr">DM4PR11MB8227.namprd11.prod.outlook.com</span> (2603:10b6:8:184::10) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 <span dir="ltr">15.20.5944.12</span>; Sun, 25 Dec 2022 12:14:07 +0000
Received: from <span dir="ltr">DM8PR11MB5653.namprd11.prod.outlook.com</span>
 ([fe80::4207:1798:c852:8fb4]) by <span dir="ltr">DM8PR11MB5653.namprd11.prod.outlook.com</span>
 ([fe80::4207:1798:c852:8fb4%6]) with mapi id 15.20.5944.016; Sun, 25 Dec 2022
 12:14:07 +0000
From: "Tyler M. Neher" &lt;<span dir="ltr">[email protected]</span>&gt;
To: "<span dir="ltr">[email protected]</span>" &lt;<span dir="ltr">[email protected]</span>&gt;
CC: Joe Ugalde &lt;<span dir="ltr">[email protected]</span>&gt;, James Westover &lt;<span dir="ltr">[email protected]</span>&gt;
Subject: request for more information
Thread-Topic: request for more information
Thread-Index: AQHZGFpiMPnsifxvPE2398qfV3yxOQ==
Date: Sun, 25 Dec 2022 12:14:06 +0000
Message-ID: &lt;<span dir="ltr">[email protected]</span>&gt;
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=<span dir="ltr">neherdata.com</span>;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DM8PR11MB5653:EE_|DM4PR11MB8227:EE_
x-ms-office365-filtering-correlation-id: 1ff1dd2e-4e05-43dd-e0da-08dae6718550
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: YZKmliwgFnMyfiKVX2l8iTYTgRsGvDlv1m7uDXv1YbJgPBw5ct9tygW6fc8nsPhCI3Uh4rZSybiPcD/DhlpUnTpzk/Hab/azB2uMsntCpuGqSejvgyS0d67pJUd+IS3o0zf+H+Qg6N4lSI2DktIsG1CslJRsS6NT4lfcckZ+3AmLMRsKucwbb3jrENebiBUCTvxB/Ugg1bBzo1eqgvzDTV/QKWomDgCQyO3RgRlBiz5WHK4Ud5Z0YmDD5HWlo1z8pZ1DFiqaeMwYYvNJAzNe+uAQzp1rlV89o1qazuVdrs7JBz9jJPuNYo3JeImEKBICoXgzSMuJUbz/frEM350b0CvI+XWjKrniWwyfDrYDDzkXo+6l+cxVAe6xMfSanIMzO+SkUWeZcOygitKO1HB9U8minbqK9ojvJFWAsUZM4LndYT7hykkZBCzgEl1WkIQfZi0snVGpUd10DueSVV8lEjWxKa2Af7lm+Fc2ieHF2vDyXVMoKPDUjQmON59lKaOsjCAVtiDQACqYZ8zwtNoyUvv4nyMC/6JyzHBzhtz/faPyiGRsvS3APFDsLpzDFIlKZqFTS34SisSbjA939WDMiG5LOXUos4NXhla1nlHRyyCKtDDOP1pjG4rZA5lWc2IPC471kwd1XHa8EsYD4rQkatdFWQB6rPsg0188zS95u8YQe6KOCNp0XD61vBR1ep6NRcxBTXUlvBv/SDcp5giwLItQgyVST0u8i+N34sp79qg=
x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:<span dir="ltr">DM8PR11MB5653.namprd11.prod.outlook.com</span>;PTR:;CAT:NONE;SFS:(<span dir="ltr">13230022</span>)(<span dir="ltr">4636009</span>)(376002)(136003)(346002)(366004)(<span dir="ltr">39830400003</span>)(396003)(<span dir="ltr">451199015</span>)(<span dir="ltr">45080400002</span>)(<span dir="ltr">66946007</span>)(<span dir="ltr">76116006</span>)(<span dir="ltr">4326008</span>)(<span dir="ltr">66556008</span>)(<span dir="ltr">66476007</span>)(<span dir="ltr">91956017</span>)(<span dir="ltr">8676002</span>)(<span dir="ltr">6916009</span>)(<span dir="ltr">2616005</span>)(<span dir="ltr">64756008</span>)(<span dir="ltr">38100700002</span>)(<span dir="ltr">54906003</span>)(316002)(<span dir="ltr">36756003</span>)(<span dir="ltr">3480700007</span>)(<span dir="ltr">2906002</span>)(<span dir="ltr">86362001</span>)(<span dir="ltr">8936002</span>)(<span dir="ltr">122000001</span>)(<span dir="ltr">83380400001</span>)(<span dir="ltr">33656002</span>)(<span dir="ltr">5660300002</span>)(<span dir="ltr">4744005</span>)(<span dir="ltr">41300700001</span>)(<span dir="ltr">40140700001</span>)(<span dir="ltr">66446008</span>)(<span dir="ltr">478600001</span>)(<span dir="ltr">6506007</span>)(<span dir="ltr">71200400001</span>)(166002)(<span dir="ltr">6486002</span>)(<span dir="ltr">107886003</span>)(186003)(<span dir="ltr">6512007</span>)(<span dir="ltr">38070700005</span>)(<span dir="ltr">45980500001</span>);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?WkE0YXp0WWtTL1BwR3pYS1ZQejFYSEFYV0VkejJWYWlZWEZ3b003c1pjUWhT?=
 =?utf-8?B?YjdyWnFrN1BCTTJTVGs0WXZpeS9BQ2ZsYXU5am8wdGtURlZ5a2pqbGtFU1NB?=
 =?utf-8?B?T2xmOTBhdXRhUjl2RlpnMnBWSVJieXpTZWNuYVh3dVRwVWJUTWt2RjE1Z1NW?=
 =?utf-8?B?WmFlQWFmN0JrYllDT2d1S2ZhK3NCWWRNemVPb1h4WDB5WkRFanEwOWZxaEFN?=
 =?utf-8?B?SS9ucXkrcGNodWl6RFM3RWhFM0NlS3pCbjcwQThtWVhPV21TMVlDampVQ0F3?=
 =?utf-8?B?dU1mTEVRcHg4Qi9RRTFIUmpvWm9VaUFVeGtPY09zRmRXYlg1VFQvWXVZWmgy?=
 =?utf-8?B?WTZoeUZsQ1gxbXpSQ1ZtTXROYU0wcU5uU2dWRkNXWGtZSk1QaHZwTG5PdHBG?=
 =?utf-8?B?cklxcENjbCt2YzhHeXhxM3d3dVBNdEIzNEJ4dnJQOTNFeVJnUXhOcTVsUkdZ?=
 =?utf-8?B?T2p6ZlpxV25iY002a3gySWU1eWpNWEVSM2RrZ21Ka0FFcEVST2thejNVTE40?=
 =?utf-8?B?dEpSb0VTRDA3aUR6S2NlMnZjN1ZsNGJvejYrQy9uakZLM2lLVDMyRC81ekhh?=
 =?utf-8?B?d0E5RWFndE9RdnhYOWpxQ0F4Y2lNYW9jTHFzMHpsRmZPdVNta0ZzUjEyY3oz?=
 =?utf-8?B?dGpvbUhmYzZZTGhUS3QyUXdmOEgzL2MzRUt0ZDMvemdaVjZWZTN1b0ZVMXor?=
 =?utf-8?B?dXRYSjczMWU4TW1Ea1dKQ2FjQ2RBaVlkNEZQTnFGM1N1NmdoY3hDNDRvaG9J?=
 =?utf-8?B?L2ZpNHdzcnBoMUQ0aTJGLytGRlY5YndhT2ZvMEVlRHJiWVNSWXJtUHZCdkgr?=
 =?utf-8?B?UHdsZ0ZCV3RkNXdoYmtHUUtVVFhNTnJ0RFY5ZDJyMUxOVThEOG85Q1dUanZr?=
 =?utf-8?B?a2Irb1ljNkFFTTdQT3h3QzlPeDI3WHVnUzczUk5oZm5KaVVQUHdZZ0k0aGoy?=
 =?utf-8?B?bDhnRWFPeFZnajNienpnM2wrZE9UQktXWHlEUXlGdjlMNDJvUEVZZmVrUDA0?=
 =?utf-8?B?TG9VWnMrTVBKeVB1SGtoUlFEMG42ZHRHZ0I1WEF1YTd5Y2krUlBtekJNLzUv?=
 =?utf-8?B?aXhWSnVXNUI5c1VjRkQwcmlrUStjRGF3NitZbUpsaFZRamVENGdHYzh0aXpi?=
 =?utf-8?B?dEZiL2NqQjJETWxvVEIzMjJRMExHMHJ5QzVQUVYwdTQwbWxWWGdGQjVUU3hE?=
 =?utf-8?B?UmRaS3BFMHhlNU9PdG55dUp4ZlM4V3Rqd1paZXRuYnZrZnhKR1BHZE5oT0dG?=
 =?utf-8?B?UGlzc2NoelNFUkllQzZsUEtPbUtsTlBZQXZraTRJMXpnRyttdTY1VWFJR0Fh?=
 =?utf-8?B?RHJHbzFkVmdXK3Y5ays3ZnhJR3lvS3U3RElsZEFqbWlKRUxzVWQrcG5sVTQv?=
 =?utf-8?B?bE5lQU82bFFUTWczdU8rbFJDeVBtUWZuOE9nY09YUGlQNi9qZkVVNWt3dFp2?=
 =?utf-8?B?cVhVZVJsM3Nmd1kxU0NqbXVpOFhWbDBCRmRMQUIrb1RnalZnbE96VW5aMWxu?=
 =?utf-8?B?cTJ1dTgwNUZpQ21ZUDNNMVlkTUN6R1JBUTFya2o4ZkRpK2h1OStrRFZtc3dO?=
 =?utf-8?B?L1kvdW13eGVMRDhkRUlTUktLbDIzTlZBL2F0cWNsTS93eU5wTUQyMmZ2OTFR?=
 =?utf-8?B?a05HK1FMVyt0RXJPeFV3dDc2S0VOYXVvTi9DUHpDTTRxNmxoc0U4ZzZVSUxB?=
 =?utf-8?B?aC9SZE82UU1yV1pGbFBlNTlRRm03Z1h5M3VQOFpPUFlBT2VhU2lYTndWZUE3?=
 =?utf-8?B?d0hRa21yTW83dmlWWkpwOTlTS1BPMlJ5Zkt1WEsrTWJjRGpSUjNDa3paWjBn?=
 =?utf-8?B?VnZqUWVBVFprbmYwRE9TYlg1VTBHZHJ0UHI2UDVlRzZLekZJUjRsdXZXN2F6?=
 =?utf-8?B?b2tRcXRLc0hHaHA4M0w0UXdLNzIzYktubFZ6L1dZdUJBSlV2R2wzSHZ0cmF4?=
 =?utf-8?B?WWx6cTVMY1V4VVp4WVJXSWhzTEU2YzFRVk00YTFLdGQ4b2tRQzJYVUhUeTJM?=
 =?utf-8?B?WnE2WEhhdGxidHJTcW5HNEdGR0lwNVZqVzhsbUptazFCdkFhVWk4RStiZzV6?=
 =?utf-8?B?NTk3TU9VekQ1clRhMWVoelptL3pOSjUremxlRFU2ZStORENTRE5rV3lZRnJ3?=
 =?utf-8?B?cG9YeWNVWkhNaUFhUE15WlljTjBVL2QwTFg2M3dPczdCckNkMHRMMUExSTFN?=
 =?utf-8?Q?r3Inbx9qRUXmDfax0qsgmCyJ6KCK2cNXmb2o/u9UYT84?=
Content-Type: multipart/alternative;
	boundary="_000_67807CC935D344F1A44BC5545A899C7Eneherdatacom_"
MIME-Version: 1.0
X-OriginatorOrg: <span dir="ltr">neherdata.com</span>
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: <span dir="ltr">DM8PR11MB5653.namprd11.prod.outlook.com</span>
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ff1dd2e-4e05-43dd-e0da-08dae6718550
X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Dec 2022 12:14:06.8085
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: d770fcbe-d7dc-4cf8-8745-7d857be6c7ac
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: IUWziK5r3bB8XlnOqS0tMD9srxYnqlqt9htEIkfA0vOs1OON2QBh5B8eNP44H9gkn8zogefDMY6jPAir3D8Rmw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB8227
</pre><br class="Apple-interchange-newline" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-tap-highlight-color: rgba(26, 26, 26, 0.3); -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"><br class="Apple-interchange-newline">

Screenshots

Desktop (please complete the following information):

• OS: Mail for iOS
• Version: iOS 16.2 (20C65)

Additional context

• sent from [email protected] which uses Microsoft 365 for email

Hi,

after successfully running the requirements, and loading up the counterfeit environment, I receive the following error when trying to run counterfit.py

'ImportError: cannot import name '_status_message' from 'scipy.optimize.optimize'

Looking to try and fix this and get counterfit running again. This is on a mac. I've tried different versions of scipy, but can't find one that will work with this.

Thanks!

Trying to use some of the demos on counterfit, but running into a problem when trying:

interact creditfraud.
zsh: segmentation fault python counterfit.py

interact works for digits_blackbox, digits_keras, movie_reviews and satellite. Just specifically a problem with creditfraud.

Can you please mention on what basis the parameter ranges are considered for attacks( for hyperopt)

When attempting to scan using the -a option with multiple attacks, the script only launches the last attack.

I am using a tensorflow model trained on mnist. I've attached images displaying the issue.
The issue occurs with both the "image" and "numpy" model_data_type.

The problem identified in https://github.com/Azure/counterfit/wiki/Abusing-ML-model-file-formats-to-create-malware-on-AI-systems:-A-proof-of-concept
has been assigned CVE-2024-3660. The vulnerability note on the topic https://www.kb.cert.org/vuls/id/253266 references the wiki page.
Suggest updating the documentation and/or the wiki page to include the CVE ID, as appropriate.

Thanks!

Using the API, there's not a good way to view options..or set logging. I wrote the code and I can't remember how. Probably not good.

cfattack = Counterfit.build_attack(
    target=target,
    attack="hop_skip_jump"
)

cfattack.options ???

Add a get_options call to CFAttack?

Hello, I've just started using this library and things were working for a few days, but I think there's been an API change in TextAttack which broke it (ART works OK).

When I try to load textattack, it can't find some embeddings files:

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/alzotolalgo.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/bae.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/bert.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/camrwr.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/deepwordbug.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/improved_ga.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/InputReduction.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/Kuleshov2017.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/pwws.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/textbugger.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/textfooler.py: Could not find word_embeddings\paragramcf on server.

[+] Framework loaded successfully!

It finishes by saying the library loaded successfully, but list attacks gives

[!] No frameworks have been loaded. Try 'load '.

It was working at least 24 hours ago, so it looks like it might be the most recent TextAttack commit.

Hello !

• While trying to use counterfit, more specifically the 'new' command, i get the following error : 'EXCEPTION of type 'AttributeError' occurred with message: type object 'Config' has no attribute 'targets_path'', which is due to a variable being named targets_dir in '/examples/terminal/core/config', while being referred to as targets_path in 'examples/terminal/commands/new.py'. I wanted to point it out as it might be the case in other files and saw no mention of it anywhere.

• Whenever i try to run the "predict" command, i get the following error message : "EXCEPTION of type 'FileNotFoundError' occurred with message: [Errno 2] No such file or directory: 'counterfit/targets'", while everything else seems to work... I don't know if it's a setup problem or a path problem. Even when i follow the available demos step by step.

• I also think the MNIST tuto available in the wiki doesn't work anymore ?

The link to Defensive Guidance does not work

Hello @moohax ,
I found in your wiki that evasion attack attempts to alter inputs such that the model gives an incorrect output. However, when I trying with evasion attack hop skip jump, I found that ['initial']['input'] and ['final']['input'] are the same, I thought they should be different from each other right? Additionally, although the two inputs are the same, ['initial']['output'] and ['final']['output'] are different from each other, could you please tell me the reason? I cannot find the place where ['initial']['output'] and ['final']['output'] comes from. Thank you.

When I type in predict command.
The error shown
mnist> predict

[!] No index sample, setting random index.

EXCEPTION of type 'ValueError' occurred with message: empty range for randrange() (0,0, 0)
[!] To enable full traceback, run the following command: 'setg debug true'

the sample of my code.
C:\Users\Eng Soon Cheah\Desktop\counterfit-main\counterfit\commands\new.py

import questionary
import os
import importlib
import cmd2
import inspect
import argparse

from counterfit.core.state import CFState
from counterfit.core import config

parser = argparse.ArgumentParser()
# including this print the correct help message


def nix_path(path):
    new_path = "/".join(path.split("\\"))
    return new_path

@cmd2.with_argparser(parser)
@cmd2.with_category("Counterfit Commands")
def do_new(self, args):
    """Optional wizard to aid in creating a new attack target."""
    model_name = questionary.text("Target name:").ask()
    model_name = model_name.replace(" ", "")

    available_frameworks = list(CFState.get_instance().loaded_frameworks.keys())
    framework_choice = questionary.select("Which framework?", choices=available_frameworks).ask()

    if "textattack" in framework_choice:
        framework = "TextTarget"
    elif "art" in framework_choice:
        framework = "ArtTarget"
    else:
        raise ValueError("invalid framework")

    if framework == "TextTarget":
        model_data_type = "text"
    elif framework == "ArtTarget":
        model_data_type = questionary.select("What data type?", choices=["numpy", "image"]).ask()
    else:
        raise ValueError("invalid framework")

    if model_name not in os.listdir(config.targets_path):
        try:
            os.mkdir(f"{config.targets_path}/{model_name}")
            open(f"{config.targets_path}/{model_name}/__init__.py", "w").close()
            with open(f"{config.targets_path}/{model_name}/{model_name}.py", "w") as f:
                f.write(
                    f"""

# Generated by counterfit #

import pickle
import numpy as np
from counterfit.core.targets import ArtTarget


class Mnist(ArtTarget):
    model_name = "mnist"
    model_data_type = "image"
    model_endpoint = "counterfit/targets/tutorial/mnist_sklearn_pipeline.pkl"
    model_input_shape = (1, 28, 28)
    model_output_classes = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]
    X = []

    def __init__(self):
        self.clip_values = (0, 255)
        with open(self.model_endpoint, "rb") as f:
            self.model = pickle.load(f)

        self.data_file = "counterfit/targets/tutorial/mnist_784.npz"
        self.sample_data = np.load(self.data_file, allow_pickle=True)

        self.X = self.sample_data["X"]

    def __call__(self, x):
        scores = self.model.predict_proba(x.reshape(x.shape[0], -1))
        return scores.tolist()
"""
                )

            CFState.get_instance().import_targets()
        except Exception as e:

            self.pwarning(f"\n [!] Failed to write target file: {e}.\n")

    else:
        self.pwarning(f"\n [!] {model_name} already exists. Choose a new name.\n")

Hi, i'm install counterfit with conda .. but when i creating new targets .. i have this fail:

Traceback (most recent call last):
File "/home/kali/miniconda3/envs/counterfit/lib/python3.8/site-packages/cmd2/cmd2.py", line 2385, in onecmd_plus_hooks
stop = self.onecmd(statement, add_to_history=add_to_history)
File "/home/kali/miniconda3/envs/counterfit/lib/python3.8/site-packages/cmd2/cmd2.py", line 2836, in onecmd
stop = func(statement)
File "/home/kali/miniconda3/envs/counterfit/lib/python3.8/site-packages/cmd2/decorators.py", line 379, in cmd_wrapper
return func(*args_list, **kwargs) # type: ignore[call-arg]
File "/home/kali/miniconda3/envs/counterfit/lib/python3.8/site-packages/examples/terminal/commands/init.py", line 54, in do_new
new_.new_cmd(args)
File "/home/kali/miniconda3/envs/counterfit/lib/python3.8/site-packages/examples/terminal/commands/new.py", line 35, in new_cmd
if target_name not in os.listdir(Config.targets_path):
AttributeError: type object 'Config' has no attribute 'targets_path'
EXCEPTION of type 'AttributeError' occurred with message: type object 'Config' has no attribute 'targets_path'

Several issues were faced while trying out counterfit.

Upon starting counterfit and providing the command new -n test, an error 'AttributeError' was encountered.

• new.py referenced an attribute named targets_path in config.py but config.py does not seem to have an attribute named targets_path declared or defined
• This was resolved by adding a line to declare an attribute named targets_path with the path directed to the "targets" directory in config.py

The above issue appears to be the same issue as the first bullet point in issue #82

After resolving this issue, several other issues of similar nature were encountered:

• new.py referenced add_targets() in state.py but state.py do not any method named add_targets() declared or defined
• new.py referenced load_targets() in state.py but state.py do not any method named load_targets() declared or defined

After resolving the above issues, another exception named 'TypeError' was encountered with the message: 'NoneType' object is not callable. This issue was traced back to be from line 77 of new.py.

Other than the new command, the command predict -i range(10) (as mentioned in DEMO1-satellite.md) also does not produce the same output as documented. This appears to be the same issue as the second bullet point in issue #82

gives error saying

/usr/bin/pip3 install ".[dev]"

        clang: numpy/core/src/multiarray/conversion_utils.c
        clang: error: the clang compiler does not support 'faltivec', please use -maltivec and include altivec.h explicitly
        clang: error: the clang compiler does not support 'faltivec', please use -maltivec and include altivec.h explicitly
        clang: error: the clang compiler does not support 'faltivec', please use -maltivec and include altivec.h explicitly
        clang: error: the clang compiler does not support 'faltivec', please use -maltivec and include altivec.h explicitly

...
...
clang: error: the clang compiler does not support 'faltivec', please use -maltivec and include altivec.h explicitly
error: Command "clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DNO_ATLAS_INFO=3 -DHAVE_CBLAS -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/umath -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/npymath -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/common -Inumpy/core/include -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/include/numpy -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/include/python3.9 -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/common -Ibuild/src.macosx-10.9-universal2-3.9/numpy/core/src/npymath -c build/src.macosx-10.9-universal2-3.9/numpy/core/src/multiarray/einsum.c -o build/temp.macosx-10.9-universal2-3.9/build/src.macosx-10.9-universal2-3.9/numpy/core/src/multiarray/einsum.o -MMD -MF build/temp.macosx-10.9-universal2-3.9/build/src.macosx-10.9-universal2-3.9/numpy/core/src/multiarray/einsum.o.d -faltivec -I/System/Library/Frameworks/vecLib.framework/Headers" failed with exit status 1
[end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for numpy
  Failed to build numpy
  ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
  [end of output]

Hi all,

I am writing to you regarding an issue I have been facing while using the Counterfit framework. Firstly, I want to commend you on developing such a powerful tool to test the security robustness of models. Your efforts in creating this framework are much appreciated.

I have been trying to import a new model/target inside the Counterfit framework and although I have successfully added the model and predicted for an image, I cannot seem to use an attack on them. I have been following the instructions provided in the documentation and on your GitHub, but I have not been able to resolve the issue.

I use the the v1.1.0 version using the docker environment thanks of the Dockerfile in the repo. Here the model I try to import onto the framework :

import numpy as np
from tensorflow.keras.models import load_model

class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']

class SigmaNet:
    def __init__(self):
        self.name = 'sigmanet'
        self.model_filename = 'sigmanet.h5'
        try:
            self._model = load_model(self.model_filename)
            print('Successfully loaded', self.name)
        except (ImportError, ValueError, OSError):
            print('Failed to load', self.name)

    def color_process(self, imgs):
        if imgs.ndim < 4:
            imgs = np.array([imgs])
        imgs = imgs.astype('float32')
        mean = [125.307, 122.95, 113.865]
        std = [62.9932, 62.0887, 66.7048]
        for img in imgs:
            for i in range(3):
                img[:, :, i] = (img[:, :, i] - mean[i]) / std[i]
        return imgs

    def predict(self, img):
        processed = self.color_process(img)
        return self._model.predict(processed)

    def predict_one(self, img):
        confidence = self.predict(img)[0]
        predicted_class = np.argmax(confidence)
        return class_names[predicted_class]

I created a new CFTarget class to be able to use it into counterfit. Here the code :

import numpy as np
from tensorflow.keras.models import load_model
from counterfit.core.targets import CFTarget
import cv2


class Htbdog(CFTarget):
    data_type = "image"
    target_name = "htbdog"
    endpoint = "/tmp/sigma/sigmanet.h5"
    data_path = "/tmp/sigma/dog.png"
    input_shape = (32, 32,3)
    output_classes = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
    classifier = "closed-box"
    X = []

    def load(self):
        self.model = load_model(self.endpoint)
        img = cv2.imread(self.data_path)
        #self.X.append(img.reshape(3,32,32))
        self.X = np.array([img]).astype('float32') / 255
        print(self.X.shape)


    def predict(self, x):
        #print(x,x.shape)


        confidence = self.model.predict(x[0].reshape(-1,32,32,3)*255)
        predicted_class = np.argmax(confidence)
        #print(confidence, predicted_class, self.output_classes[predicted_class] )
        #return self.output_classes[predicted_class]
        return confidence

So as said before, I have succeed to predict with the image send into the docker environnement; But when i try to do the hop skip jump attack I got this error :

You'll find attached the model used, just remove the .txt
sigmanet.h5.txt

I have found it difficult to find additional resources online that could help me troubleshoot the problem. Most of the resources available seem to be outdated and do not provide enough guidance on how to solve this particular issue. I would really appreciate it if you could provide me with some guidance or assistance in resolving this issue. Is there any additional documentation or resources that I can refer to? Also, could you please provide me with some steps or guidelines on how to import a new model/target and use an attack on them?

Branch: In develop 1.1
Attack: hop_skip_jump

The ART estimator and ART attacks have different parameters, each are set with their set_params function. However, the ART framework in Counterfit only sets the parameters on the attack and not the estimator. Not all options are being set properly.

cfattack.options.update({
    "clip_values": 128
})

# Is None.
cfattack.attack.clip_values 

It seems to be this issue. QData/TextAttack#424

As a workaround I started counterfit in WSL, but it looks like the issue has been patched in textattack so if you can pull in the fix that would be great.

Traitlets overwrites an init import that causes Counterfit import in Jupyter in VSCode to fail. Solution has been to pip uninstall traitlets

counterfit> set_target digits_mlp
digits_mlp> set_attack newtonfool
[+] success: Using 0e34bf51
digits_mlp>NewtonFool:0e34bf51> run
NewtonFool: 0%| | 0/1 [00:00<?, ?it/s]
[!] failed: Failed to run 0e34bf51 (art.attacks.evasion.newtonfool.NewtonFool): 'BlackBoxClassifier' object has no attribute
'class_gradient'

Branch: develop/1.1

Getting samples happens in cfattack.__init__. It should happen in cfattack.prepare_attack which is called before the attack is run in core.run_attack.

        # Get the samples.
        self.samples = target.get_samples(
            self.options.cf_options["sample_index"]["current"]
        )

[email protected] email does not work. I get this error:

Message blocked

Your message to [email protected] has been blocked. See technical details below for more information.

I have also confirmed this with Aasim at MS Azure support and you may be hearing from him, too. He suggested I enter this here, too.

I was going to ask in the email where I can get more information on the Attack Parameters. I have checked the wiki and other resources and I seem to be unable to find more. Thank you for anything you can do here. My email is [email protected] if you want to reach me that way. Thank you for a great tool and I would enjoy contributing.

Paul Starrett

Describe the bug
While in Counterfit conda env, I attempted to invoke 'python examples/terminal/terminal.py' but get error that Torch has no module 'nn'. I also get this error when running the code in Python (where I import counterfit and run the sample code). See Additional context below.

POSSIBLE FIX: I see the below statement in another app I am using. The error in this issue may be due to not creating the import alias (?):

import torch.nn as nn

To Reproduce
Steps to reproduce the behavior:

1. Go to https://github.com/Azure/counterfit/blob/main/examples/DEMO2-creditfraud.md
2. Run 'python examples/terminal/terminal.py'
3. The stack trace listed below occurs.
4. Or, run the sample code where imports are performed in PY file and attacks are called in PY file (see Additional context).

Expected behavior
The terminal should allow use of the creditfraud shell as below
counterfit> set_target creditfraud
creditfraud>

Stack trace
(counterfit) paul@paul-ThinkPad-T480s:~/Desktop/Counterfit-AML/counterfit$ python examples/terminal/terminal.py
Traceback (most recent call last):
File "examples/terminal/terminal.py", line 9, in
from examples.terminal.core.terminal import Terminal
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/examples/terminal/core/terminal.py", line 6, in
from counterfit import CFPrint
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/counterfit/init.py", line 9, in
from . import core, data, frameworks, reporting, targets
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/counterfit/frameworks/init.py", line 5, in
from .art import art
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/counterfit/frameworks/art/art.py", line 17, in
from art.utils import compute_success_array, random_targets
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/init.py", line 7, in
from art import attacks
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/attacks/init.py", line 8, in
from art.attacks import evasion
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/attacks/evasion/init.py", line 4, in
from art.attacks.evasion.adversarial_patch.adversarial_patch import AdversarialPatch
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/attacks/evasion/adversarial_patch/adversarial_patch.py", line 31, in
from art.attacks.evasion.adversarial_patch.adversarial_patch_numpy import AdversarialPatchNumpy
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/attacks/evasion/adversarial_patch/adversarial_patch_numpy.py", line 37, in
from art.estimators.estimator import BaseEstimator, NeuralNetworkMixin
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/estimators/init.py", line 17, in
from art.estimators import certification
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/estimators/certification/init.py", line 9, in
from art.estimators.certification import deep_z
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/estimators/certification/deep_z/init.py", line 4, in
from art.estimators.certification.deep_z.deep_z import ZonoDenseLayer
File "/home/paul/anaconda3/envs/counterfit/lib/python3.8/site-packages/art/estimators/certification/deep_z/deep_z.py", line 29, in
class ZonoDenseLayer(torch.nn.Module):
AttributeError: module 'torch' has no attribute 'nn'

Screenshots
N/A

Desktop (please complete the following information):

• OS: Linux Ubuntu
• Version: 22.04.1 LTS

Additional context
I followed install instructions from current site (today). I was using the older version where counterfit.py was used (if that helps) I installed into a completely new dir, conda env, etc.

I also get this error when running the below code (from https://github.com/Azure/counterfit):

import counterfit
import counterfit.targets as targets

target = targets.CreditFraud()
target.load()
attack_name = 'hop_skip_jump'
new_attack = counterfit.Counterfit.build_attack(target, attack_name)
results = counterfit.Counterfit.run_attack(new_attack)

Branch: develop/1.1

self.cf_options[k]["previous"].append(self.cf_options[k]["current"])
KeyError: 'previous'

caused by logger options not having a previous key,

Fix:

"logger": {
    "default": "basic",
    "current": "basic",
    "previous": [],
    "docs": "Logger to log queries with"
}

Hello @moohax,

I found that I cannot load the framework info from Config.json when loading framework. Could you please tell me how to deal with this problem? Thank you.

Running the moviereviews tutorial (interact moviereviews, use deepwordbug, run) throws the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/cmd2/cmd2.py", line 2063, in onecmd_plus_hooks
stop = self.onecmd(statement, add_to_history=add_to_history)
File "/usr/local/lib/python3.7/dist-packages/cmd2/cmd2.py", line 2493, in onecmd
stop = func(statement)
File "/usr/local/lib/python3.7/dist-packages/cmd2/decorators.py", line 318, in cmd_wrapper
return func(*args_list, **kwargs)
File "/content/counterfit/commands/run.py", line 56, in do_run
CFState.get_instance().active_target.run_attack(logging=args.log)
File "/content/counterfit/core/targets.py", line 194, in run_attack
resulting_samples = self._run_attack(logging) # call the specific method
File "/content/counterfit/core/targets.py", line 306, in _run_attack
return self._run_textattack_attack(logging)
File "/content/counterfit/core/targets.py", line 296, in _run_textattack_attack
results_iter = attack_cls.attack_dataset(
AttributeError: 'Attack' object has no attribute 'attack_dataset'
EXCEPTION of type 'AttributeError' occurred with message: 'Attack' object has no attribute 'attack_dataset'

I get the same error when I try with a different text classifier and dataset, so I think it might be one of the recent TextAttack commits? It ran fine until the first Friday commit. ART still loads and works fine.

Hello!

I have been enjoying trying out counterfit but I was wondering if there is (a plan to) support batch processing?

Many thanks

I installed Counterfit using Option 2. While I am going through 'Basic Use' example, I got this Exception. Any suggestions.

File "C:.....\Anaconda3\envs\counterfit\lib\site-packages\art\utils.py", line 534, in check_and_transform_label_format
if len(labels.shape) == 2 and labels.shape[1] > 1:
AttributeError: 'list' object has no attribute 'shape'
EXCEPTION of type 'AttributeError' occurred with message: 'list' object has no attribute 'shape'

Is not working for me.
Is there another way to launch this on Azure ?

Thank you

There is a critical dependancy in the dependabot findings around tensorflow that I would strongly suggest you fix. It seems that the fix is a non-breaking fix however, you will know better.

Just a thought.

Yours,

Ken

Hi

I get a permission error during the run command for hop_skip_jump in the Azure deployment/Azure Shell. How to resolve this?

satelliteimages>hop_skip_jump> run

[+] Running hop_skip_jump on satelliteimages

HopSkipJump: 100%|████████████████████████████████| 1/1 [00:07<00:00, 7.40s/it]
EXCEPTION of type 'PermissionError' occurred with message: [Errno 13] Permission denied: 'counterfit/targets/satelliteimages/results'

Thanks !

Problem while creating a new target inside a counterfit docker container:

[!] Failed to write target file: [Errno 13] Permission denied: 'counterfit/targets/dummy'.

My container is based on the image pulled using docker pull azuretrustworthyml/counterfit:0.1

Any help to fix this please..

Building wheels for collected packages: python-Levenshtein, torchfile, bs4, gym, networkx, clang, future, iopath, pyperclip, termcolor, wrapt, gdown, mpld3, word2number, docopt, sqlitedict, langdetect, wikipedia-api, overrides
Building wheel for python-Levenshtein (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [31 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-38
creating build/lib.linux-x86_64-cpython-38/Levenshtein
copying Levenshtein/StringMatcher.py -> build/lib.linux-x86_64-cpython-38/Levenshtein
copying Levenshtein/init.py -> build/lib.linux-x86_64-cpython-38/Levenshtein
running egg_info
writing python_Levenshtein.egg-info/PKG-INFO
writing dependency_links to python_Levenshtein.egg-info/dependency_links.txt
deleting python_Levenshtein.egg-info/entry_points.txt
writing namespace_packages to python_Levenshtein.egg-info/namespace_packages.txt
writing requirements to python_Levenshtein.egg-info/requires.txt
writing top-level names to python_Levenshtein.egg-info/top_level.txt
reading manifest file 'python_Levenshtein.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*pyc' found anywhere in distribution
warning: no previously-included files matching '*so' found anywhere in distribution
warning: no previously-included files matching '.project' found anywhere in distribution
warning: no previously-included files matching '.pydevproject' found anywhere in distribution
adding license file 'COPYING'
writing manifest file 'python_Levenshtein.egg-info/SOURCES.txt'
copying Levenshtein/_levenshtein.c -> build/lib.linux-x86_64-cpython-38/Levenshtein
copying Levenshtein/_levenshtein.h -> build/lib.linux-x86_64-cpython-38/Levenshtein
running build_ext
building 'Levenshtein._levenshtein' extension
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/Levenshtein
gcc -pthread -B /home/mike/anaconda3/envs/counterfit/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/mike/anaconda3/envs/counterfit/include/python3.8 -c Levenshtein/_levenshtein.c -o build/temp.linux-x86_64-cpython-38/Levenshtein/_levenshtein.o
error: command 'gcc' failed: No such file or directory
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for python-Levenshtein
Running setup.py clean for python-Levenshtein
Building wheel for torchfile (setup.py) ... done
Created wheel for torchfile: filename=torchfile-0.1.0-py3-none-any.whl size=5693 sha256=9ffaf688f6b324ff14fd5d40238bae410d91407f0b3f5135f0fe0aa76a4e3f75
Stored in directory: /home/mike/.cache/pip/wheels/b9/99/d1/9f3f4411a958a22ccf782e33c7238a07f04a9597f2f3b38792
Building wheel for bs4 (setup.py) ... done
Created wheel for bs4: filename=bs4-0.0.1-py3-none-any.whl size=1257 sha256=cd3f60c49624cadc73dfc3f026ddfd440a4f79a1e05f3d6c6c7945ad31a21540
Stored in directory: /home/mike/.cache/pip/wheels/75/78/21/68b124549c9bdc94f822c02fb9aa3578a669843f9767776bca
Building wheel for gym (setup.py) ... done
Created wheel for gym: filename=gym-0.19.0-py3-none-any.whl size=1663091 sha256=027f153e5fdc6342102d2b0bb5c9d103d27a543c91a640f546a344b3b8a52e88
Stored in directory: /home/mike/.cache/pip/wheels/11/36/28/628f4dd3779e4037a6fca1aaed76827ffa4315c3ab6bfadcf6
Building wheel for networkx (setup.py) ... done
Created wheel for networkx: filename=networkx-2.2-py2.py3-none-any.whl size=1526919 sha256=cd2e280d8fd8582c43bc79c94485f58c4feb27b4caa2ab970623c5a16ebf4074
Stored in directory: /home/mike/.cache/pip/wheels/c3/d7/dd/7e2e485ecd56541b03075c9e3692ec12f6b43e90d09f9a6a9a
Building wheel for clang (setup.py) ... done
Created wheel for clang: filename=clang-5.0-py3-none-any.whl size=30681 sha256=0bc080e6b00b857ac4babf8c1dcb6a5021f7295ab6d5c9cc7201c2b3b50d7876
Stored in directory: /home/mike/.cache/pip/wheels/f1/60/77/22b9b5887bd47801796a856f47650d9789c74dc3161a26d608
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 sha256=578710c9de7bdef01072479ce01ef0c22685cf370074e6572b801d96e661c511
Stored in directory: /home/mike/.cache/pip/wheels/8e/70/28/3d6ccd6e315f65f245da085482a2e1c7d14b90b30f239e2cf4
Building wheel for iopath (setup.py) ... done
Created wheel for iopath: filename=iopath-0.1.10-py3-none-any.whl size=31532 sha256=61864d9531c7806b20d5d8985882381f0d69738c6ff79335e7102c952a4abf21
Stored in directory: /home/mike/.cache/pip/wheels/89/3e/24/0f349c0b2eeb6965903035f3b00dbb5c9bea437b4a2f18d82c
Building wheel for pyperclip (setup.py) ... done
Created wheel for pyperclip: filename=pyperclip-1.8.2-py3-none-any.whl size=11123 sha256=75079a19dea97a998282a18b3e98f63cc32026c1b2bcaa35858de6442e409f95
Stored in directory: /home/mike/.cache/pip/wheels/7f/1a/65/84ff8c386bec21fca6d220ea1f5498a0367883a78dd5ba6122
Building wheel for termcolor (setup.py) ... done
Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4832 sha256=afabaf04dd54fc73d2bbf7f267f9c090fe51636603bfa401270b3c49db4ec544
Stored in directory: /home/mike/.cache/pip/wheels/a0/16/9c/5473df82468f958445479c59e784896fa24f4a5fc024b0f501
Building wheel for wrapt (setup.py) ... done
Created wheel for wrapt: filename=wrapt-1.12.1-py3-none-any.whl size=19553 sha256=7c03d30f68b6abd56dfd6b224a02cc85e2a0eae57b3aae856eff32c4df0a4996
Stored in directory: /home/mike/.cache/pip/wheels/5f/fd/9e/b6cf5890494cb8ef0b5eaff72e5d55a70fb56316007d6dfe73
Building wheel for gdown (pyproject.toml) ... done
Created wheel for gdown: filename=gdown-3.12.2-py3-none-any.whl size=9674 sha256=a8f92d2e219dd25bc85f917f5e5cdbf5bfd3ba8753f8f7930868f31b119bcfae
Stored in directory: /home/mike/.cache/pip/wheels/e2/62/1e/926d1ebe7b1e733c78d627fd288d01b83feaf67efc06e0e4c3
Building wheel for mpld3 (setup.py) ... done
Created wheel for mpld3: filename=mpld3-0.3-py3-none-any.whl size=116686 sha256=b4ecb5d9673cc060d659690b720838db99cbabb4e0339a1707d9301f5aa2422e
Stored in directory: /home/mike/.cache/pip/wheels/3d/9f/9d/d806a20bd97bc7076d724fa3e69fa5be61836ba16b2ffa6126
Building wheel for word2number (setup.py) ... done
Created wheel for word2number: filename=word2number-1.1-py3-none-any.whl size=5567 sha256=c76c90012dd58a1fcf7826ad4be92b1cabfaaa88c68e56e23bc1f5d8a3b174c5
Stored in directory: /home/mike/.cache/pip/wheels/cb/f3/5a/d88198fdeb46781ddd7e7f2653061af83e7adb2a076d8886d6
Building wheel for docopt (setup.py) ... done
Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13706 sha256=ed4bd721de91451b090dd9e31ced07696d4c7cfca6445d0b3f602396d5d2e5ee
Stored in directory: /home/mike/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c
Building wheel for sqlitedict (setup.py) ... done
Created wheel for sqlitedict: filename=sqlitedict-2.0.0-py3-none-any.whl size=15734 sha256=68064c06004c819ed85ffab311942f528c86a25c7b5c5cb2d5c49d1fd226ee1e
Stored in directory: /home/mike/.cache/pip/wheels/ee/0b/8c/3cdf3e7eef4161d79c62df5bef35b0614238d0d2bd3051877a
Building wheel for langdetect (setup.py) ... done
Created wheel for langdetect: filename=langdetect-1.0.9-py3-none-any.whl size=993225 sha256=09f49cf026b6cdf7d53cde07d5cebd86c515ad7700f003405e8420dd9ee04c74
Stored in directory: /home/mike/.cache/pip/wheels/13/c7/b0/79f66658626032e78fc1a83103690ef6797d551cb22e56e734
Building wheel for wikipedia-api (setup.py) ... done
Created wheel for wikipedia-api: filename=Wikipedia_API-0.5.4-py3-none-any.whl size=13476 sha256=6df36bf132e194bb22c392f40433e64681c5df6a0c4946b250c675158304217a
Stored in directory: /home/mike/.cache/pip/wheels/ed/88/e3/da3d4d73cb91d659488cfa25913b84bbc26febec99d257bce9
Building wheel for overrides (setup.py) ... done
Created wheel for overrides: filename=overrides-3.1.0-py3-none-any.whl size=10172 sha256=49e166626d2bb5c43c287b6a3d9fea8c9ed8449049f4911ea0350198972d51c1
Stored in directory: /home/mike/.cache/pip/wheels/6a/4f/72/28857f75625b263e2e3f5ab2fc4416c0a85960ac6485007eaa
Successfully built torchfile bs4 gym networkx clang future iopath pyperclip termcolor wrapt gdown mpld3 word2number docopt sqlitedict langdetect wikipedia-api overrides
Failed to build python-Levenshtein
Installing collected packages: wrapt, word2number, wcwidth, typing-extensions, torchfile, tokenizers, termcolor, tensorflow-estimator, tensorboard-plugin-wit, sqlitedict, snowballstemmer, sentencepiece, pytz, pyperclip, pyglet, pyasn1, overrides, nlpaug, mpld3, lru-dict, keras, janome, ipython-genutils, iniconfig, flatbuffers, docopt, commonmark, clang, alabaster, zipp, xxhash, urllib3, traitlets, tqdm, torch, toml, threadpoolctl, terminaltables, tensorboard-data-server, tabulate, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinxcontrib-devhelp, sphinxcontrib-applehelp, soupsieve, smart-open, six, setuptools, rsa, regex, PyYAML, python-magic, PySocks, pyparsing, pygments, pyDeprecate, pycparser, pyasn1-modules, py, protobuf, prompt_toolkit, portalocker, pluggy, pillow, orjson, oauthlib, numpy, num2words, multidict, more-itertools, MarkupSafe, lxml, llvmlite, konoha, kiwisolver, joblib, itsdangerous, imagesize, idna, gast, future, ftfy, fsspec, frozenlist, fonttools, filelock, editdistance, docutils, dill, deprecated, decorator, cycler, conllu, colorama, cloudpickle, click, charset-normalizer, cachetools, babel, attrs, async-timeout, yarl, Werkzeug, torchvision, tensorflow-hub, segtok, scipy, rich, requests, questionary, python-Levenshtein, python-dateutil, pyarrow, packaging, opt-einsum, numba, nltk, networkx, multiprocess, lemminflect, langdetect, keras-preprocessing, Jinja2, isodate, iopath, importlib-metadata, h5py, gym, grpcio, google-pasta, google-auth, contourpy, cmd2, cffi, beautifulsoup4, astunparse, aiosignal, absl-py, wikipedia-api, torchmetrics, sphinx, scikit-learn, requests-oauthlib, pytest, pandas, matplotlib, markdown, language-tool-python, hyperopt, huggingface-hub, gensim, flask, cryptography, bs4, azure-core, augly, aiohttp, transformers, sphinx-rtd-theme, msrest, lightgbm, google-auth-oauthlib, gdown, datasets, bpemb, adversarial-robustness-toolbox, tensorboard, flair, bert-score, azure-storage-blob, textattack, tensorflow, pytorch_lightning, tensorflow-text
Attempting uninstall: setuptools
Found existing installation: setuptools 65.5.0
Uninstalling setuptools-65.5.0:
Successfully uninstalled setuptools-65.5.0
Running setup.py install for python-Levenshtein ... error
error: subprocess-exited-with-error

× Running setup.py install for python-Levenshtein did not run successfully.
│ exit code: 1
╰─> [34 lines of output]
running install
/home/mike/anaconda3/envs/counterfit/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/Levenshtein
copying Levenshtein/StringMatcher.py -> build/lib.linux-x86_64-3.8/Levenshtein
copying Levenshtein/init.py -> build/lib.linux-x86_64-3.8/Levenshtein
running egg_info
writing python_Levenshtein.egg-info/PKG-INFO
writing dependency_links to python_Levenshtein.egg-info/dependency_links.txt
writing entry points to python_Levenshtein.egg-info/entry_points.txt
writing namespace_packages to python_Levenshtein.egg-info/namespace_packages.txt
writing requirements to python_Levenshtein.egg-info/requires.txt
writing top-level names to python_Levenshtein.egg-info/top_level.txt
reading manifest file 'python_Levenshtein.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*pyc' found anywhere in distribution
warning: no previously-included files matching '*so' found anywhere in distribution
warning: no previously-included files matching '.project' found anywhere in distribution
warning: no previously-included files matching '.pydevproject' found anywhere in distribution
adding license file 'COPYING'
writing manifest file 'python_Levenshtein.egg-info/SOURCES.txt'
copying Levenshtein/_levenshtein.c -> build/lib.linux-x86_64-3.8/Levenshtein
copying Levenshtein/_levenshtein.h -> build/lib.linux-x86_64-3.8/Levenshtein
running build_ext
building 'Levenshtein._levenshtein' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/Levenshtein
gcc -pthread -B /home/mike/anaconda3/envs/counterfit/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/mike/anaconda3/envs/counterfit/include/python3.8 -c Levenshtein/_levenshtein.c -o build/temp.linux-x86_64-3.8/Levenshtein/_levenshtein.o
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> python-Levenshtein

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Environment created as per Option2 instructions. Running creditcard with art framework. Following instructions from Webinar-Demo-2.

creditfraud> setg debug true
debug - was: False
now: True
creditfraud>boundary> scan -n 3 --log

[+] Running these attacks 3x each:
boundary, zoo, hop_skip_jump

[+] Using boundary 60fb175a

Traceback (most recent call last):
File "/home/krbock/anaconda3/envs/counterfit/lib/python3.7/site-packages/cmd2/cmd2.py", line 2063, in onecmd_plus_hooks
stop = self.onecmd(statement, add_to_history=add_to_history)
File "/home/krbock/anaconda3/envs/counterfit/lib/python3.7/site-packages/cmd2/cmd2.py", line 2493, in onecmd
stop = func(statement)
File "/home/krbock/anaconda3/envs/counterfit/lib/python3.7/site-packages/cmd2/decorators.py", line 318, in cmd_wrapper
return func(*args_list, **kwargs)
File "/home/krbock/GitHub/counterfit/counterfit/commands/scan.py", line 166, in do_scan
func(self, args)
File "/home/krbock/GitHub/counterfit/counterfit/commands/scan.py", line 149, in scan_attacks_on_active_target
perform_attacks_on_target(self, attacks, num_iters, has_logging_enabled, has_verbose_enabled, set_params, class_summary)
File "/home/krbock/GitHub/counterfit/counterfit/commands/scan.py", line 105, in perform_attacks_on_target
CFState.get_instance().active_target.run_attack(logging=has_logging_enabled)
File "/home/krbock/GitHub/counterfit/counterfit/core/targets.py", line 194, in run_attack
resulting_samples = self._run_attack(logging) # call the specific method
File "/home/krbock/GitHub/counterfit/counterfit/core/targets.py", line 266, in _run_attack
return self._run_art_attack(logging)
File "/home/krbock/GitHub/counterfit/counterfit/core/targets.py", line 256, in _run_art_attack
self.active_attack.samples, [self.active_attack.target_class] * len(self.active_attack.samples)
File "/home/krbock/anaconda3/envs/counterfit/lib/python3.7/site-packages/art/attacks/attack.py", line 75, in replacement_function
return fdict[func_name](self, *args, **kwargs)
File "/home/krbock/anaconda3/envs/counterfit/lib/python3.7/site-packages/art/attacks/evasion/boundary.py", line 128, in generate
if y is not None and self.estimator.nb_classes == 2 and y.shape[1] == 1:
IndexError: tuple index out of range
EXCEPTION of type 'IndexError' occurred with message: tuple index out of range

Hi there,

I downloaded the latest version of Counterfit last week and installed all the modules required but still have some problems. When I executed the command 'load art' I got a warning:

load art

The type of the provided estimator is not yet support for automated setting of logits difference loss. Therefore, this attack is defaulting to attacking the loss provided by the model in the provided estimator.
[+] art successfully loaded with defaults (no config file provided)

which did not exist before. When I executed the 'run' command I only got the adversarial input:

run

[-] Running attack HopSkipJump with id 12a70390 on creditfraud)

[-] Preparing attack...
[-] Running attack...
┌─────────┬──────────────┬──────────────────────────┐
│ Success │ Elapsed time │ Total Queries │
├─────────┼──────────────┼──────────────────────────┤
│ 1/1 │ 4.3 │ 24550 (5740.6 query/sec) │
└─────────┴──────────────┴──────────────────────────┘
┌┬┬┬┬────────────────────────────────────────────────────────────────────────┬┐
│││││ Adversarial Input ││
├┼┼┼┼────────────────────────────────────────────────────────────────────────┼┤
│││││ [4462.00 -2.30 1.76 -0.36 2.33 -0.82 -0.07 0.56 -0.40 -0.24 -1.53 2.03 ││
│││││ -6.56 0.17 -1.47 -0.70 -2.28 -4.78 -2.62 -1.34 -0.43 -0.30 -0.93 0.17 ││
│││││ -0.09 -0.15 -0.54 0.04 -0.15 239.93] ││
└┴┴┴┴────────────────────────────────────────────────────────────────────────┴┘
[+] Attack completed 12a70390 (HopSkipJump)

there are some differences in the scan summary as well (no 'queries'):

Additionally, could you please explain some meaning of values in the scan summary and the running output? Could you please tell me what does it mean by 'successes'? When can we say an attack is a 'success' or 'failure'? I guess that 'best score' means the success percentage of samples in an attack, is it correct?

In the running output, I guess the sample index means the number of samples in the attack right? What is the meaning of label and attack label? What is the meaning of % Eucl. dist. and Elapsed Time [sec]? I think 'queries' means the number of the attack on target, is it correct? I saw a list of decimal numbers in the adversarial input value, are they only random numbers or they have some similarities? Where are they from? Do they have some relations with my attack type?

Thank you very much for your help and patience.

Branch: develop/1,1

I think a better practice is to only have it run the attack, I get issues in which attack completes but check_success fails for unrelated reasons.

Hello @moohax,

I'm sorry that I am not sure if you have seen my questions so I opened a new issue.

I am trying to add new attacks to the old version of Counterfit (There is no extraction attack and inference attack in the old version). Could you please give me some tips?

For instance, I consulted the config file in the new Counterfit version and tried to create a new file for utilizing Functionally Equivalent Extraction attack from ART in the old version. Could you please tell me if I need to add some new information in the file? Thank you.

Hi there,
very nice project, is there a plan to implement also attacks that works on the label output (no probabilities) or limited API query setting?
There was an article here a while ago which would be nice to have.
Is this something that should be implemented in the ART component?

Hi there,
Would be nice to provide some info about the credit card target example.
We know it's 30 vector size, but what was the training set and what methods was used for classification?
Cheers.